00010000:00010000:1.0:1787827240.402552:0:48370:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e42d lrc: 4/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.402560:0:48370:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e42d lrc: 5/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.402566:0:48370:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827240.402571:0:48370:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e42d lrc: 3/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.402579:0:48370:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e42d lrc: 0/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.402593:0:48370:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a67ecf00 x1874671476527104/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787827240.402603:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200001b72:0x2:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787827240.402616:0:48370:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 30 res 8589941618/2 (0 0) 00000080:00200000:1.0:1787827240.402676:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.402686:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.402693:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.402704:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.402712:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.402716:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.402719:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.402724:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e03bf8 00010000:00010000:1.0:1787827240.402728:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.402743:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.402754:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.402759:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:1.0:1787827240.402763:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.402766:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=25 00000080:00200000:1.0:1787827240.402769:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.402776:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.402779:0:48370:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=test-framework.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), file ffff9a788b0fa300, open_flags 80c1, mode 81ed opened 0 00000080:00200000:1.0:1787827240.402786:0:48370:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test-framework.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), intent=open|creat 00000002:00010000:1.0:1787827240.402800:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test-framework.sh,[0x200001b72:0x4e:0x0]) in obj [0x200001b72:0x2:0x0], intent: open|creat flags 0100302 00010000:00010000:1.0:1787827240.402824:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.402847:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787827240.402854:0:48370:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787827240.402859:0:48370:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ecf00 x1874671476527360/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787827240.402866:0:48370:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.402876:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ecf00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476527360:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.402976:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7889045240 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.404313:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ecf00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476527360:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.404325:0:48370:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 4/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404338:0:48370:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404345:0:48370:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404356:0:48370:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404363:0:48370:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404370:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404376:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404381:0:48370:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 4/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.404388:0:48370:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a67ecf00 x1874671476527360/t12884902241(12884902241) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787827240.404397:0:48370:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ecf00 x1874671476527360/t12884902241(12884902241) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827240.404412:0:48370:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787827240.404424:0:48370:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404435:0:48370:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 4 res 8589941618/78 (0 0) 00000080:00200000:1.0:1787827240.404440:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200001b72:0x2:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787827240.404449:0:48370:0:(namei.c:1628:ll_create_it()) VFS Op:name=test-framework.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), intent=open|creat 00000080:00200000:1.0:1787827240.404476:0:48370:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x4e:0x0](ffff9a78916faa08) 00000080:00200000:1.0:1787827240.404566:0:48370:0:(file.c:6854:ll_layout_conf()) [0x200001b72:0x4e:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787827240.404571:0:48370:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200001b72:0x4e:0x0](ffff9a78916faa08) 00000080:00010000:1.0:1787827240.404574:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x4e:0x0](ffff9a78916faa08) for lock 0xebdcd48a4298e434 00000080:00010000:1.0:1787827240.404580:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x4e:0x0](ffff9a78916faa08) 00000080:00200008:1.0:1787827240.404588:0:48370:0:(file.c:1090:ll_file_open()) START file test-framework.sh:[0x200001b72:0x4e:0x0](ffff9a78916faa08), flags 100301 00000002:00100000:1.0:1787827240.404608:0:48370:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ecf00 x1874671476527360/t12884902241(12884902241) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787827240.404642:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e434 from it ffff9a78908e1540 00010000:00010000:1.0:1787827240.404646:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404652:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.404666:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x4e:0x0](ffff9a78916faa08), inode mode 81ed mask 40 00000080:00200008:1.0:1787827240.404677:0:48370:0:(file.c:6313:ll_getattr_dentry()) START file test-framework.sh:[0x200001b72:0x4e:0x0](ffff9a78916faa08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827240.404682:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200001b72:0x4e:0x0](ffff9a78916faa08),name=test-framework.sh 00000002:00010000:1.0:1787827240.404691:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200001b72:0x4e:0x0]) in obj [0x200001b72:0x2:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.404697:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.404705:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.404710:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200001b72:0x2:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.404713:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x4e:0x0](ffff9a78916faa08) 00000080:00010000:1.0:1787827240.404714:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x4e:0x0](ffff9a78916faa08) for lock 0xebdcd48a4298e434 00000080:00010000:1.0:1787827240.404717:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e434 from it ffffaa5487e03cd8 00010000:00010000:1.0:1787827240.404719:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.404725:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827240.404742:0:48370:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4e:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787827240.404753:0:48370:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200001b72:0x4e:0x0] 00000008:00010000:1.0:1787827240.404770:0:48370:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787827240.404774:0:48370:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7886af01b0, osc lock ffff9a789822a138, flags 0x400001002 00010000:00010000:1.0:1787827240.404818:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002741f3ff/0xebdcd48a4298e43b lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.404826:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002741f3ff/0xebdcd48a4298e43b lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.404886:0:48370:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1787827240.405982:0:48370:0:(file.c:1775:ll_merge_attr_nolock()) [0x200001b72:0x4e:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787827240.405990:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787827240.405996:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.406003:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7886af01b0 (ffff9a7885afd488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.406008:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787827240.406013:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7886af01b0 (ffff9a7885afd7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787827240.406029:0:48370:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4e:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:1.0:1787827240.406266:0:48370:0:(file.c:2652:do_file_write_iter()) START file test-framework.sh:[0x200001b72:0x4e:0x0], ppos: 0, count: 317548 00000080:00200000:1.0:1787827240.406276:0:48370:0:(file.c:2090:ll_file_io_generic()) test-framework.sh: write ppos: 0, bytes: 317548 00000080:00200000:1.0:1787827240.406281:0:48370:0:(file.c:1848:ll_io_set_mirror()) test-framework.sh: desiginated mirror: 0 00000020:00200000:1.0:1787827240.406287:0:48370:0:(cl_io.c:218:cl_io_rw_init()) header@0000000032981a02[0x110, 1, [0x200001b72:0x4e:0x0] hash] 00000020:00200000:1.0:1787827240.406289:0:48370:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 317548) 0 0 00000080:00200000:1.0:1787827240.406292:0:48370:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4e:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.406303:0:48370:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 77] 00020000:00200000:1.0:1787827240.406312:0:48370:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 317548) 317548, 317548 00020000:00200000:1.0:1787827240.406317:0:48370:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787827240.406332:0:48370:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 77] 00000008:00010000:1.0:1787827240.406343:0:48370:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:1.0:1787827240.406349:0:48370:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7886af01b0, osc lock ffff9a789822a138, flags 0x0 00010000:00010000:1.0:1787827240.406359:0:48370:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161089/135 (0 319487) 00010000:00010000:1.0:1787827240.406384:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.406391:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.406398:0:48370:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.406414:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ef0c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476527616:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.406449:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393a80 going to sleep for 11 seconds 00010000:00010000:1.0:1787827240.407535:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.407543:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002741f3ff/0xebdcd48a4298e43b lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8daae2 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.407551:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78908c3e00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.407556:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002741f3ff/0xebdcd48a4298e43b lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8daae2 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.407571:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.407580:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.407609:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002741f3ff/0xebdcd48a4298e43b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8daae2 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.407618:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.407646:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002741f3ff/0xebdcd48a4298e43b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8daae2 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.407656:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002741f3ff/0xebdcd48a4298e43b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8daae2 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.407678:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.407681:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.407682:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.408251:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ef0c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476527616:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.408267:0:48370:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.408284:0:48370:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.408295:0:48370:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.408958:0:48370:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.408975:0:48370:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.408991:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 5/0,2 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.409001:0:48370:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,2 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.409009:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 5/0,2 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.409015:0:48370:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.409034:0:48370:0:(vvp_io.c:1339:vvp_io_write_start()) test-framework.sh: write [0, 317548) append 0 append lockpos 0 00000080:00200000:1.0:1787827240.409048:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409097:0:48370:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409110:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000fc7e2224[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 0] 00000080:00200000:1.0:1787827240.409115:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000009f59f3a5 17ffffc0001001 3:0 ffff9a788307c828 0 lru 00000080:00200000:1.0:1787827240.409125:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000fdd65128 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409129:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000fc7e2224 00000080:00200000:1.0:1787827240.409130:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:1.0:1787827240.409135:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409162:0:48370:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409168:0:48370:0:(rw26.c:933:ll_write_end()) page@000000004ed71c9e[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 1] 00000080:00200000:1.0:1787827240.409172:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000062a5e809 17ffffc0001001 3:0 ffff9a788307c000 1 lru 00000080:00200000:1.0:1787827240.409179:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000005f3f2bd2 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409181:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000004ed71c9e 00000080:00200000:1.0:1787827240.409183:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:1.0:1787827240.409185:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409201:0:48370:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409204:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000cb7301cc[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 2] 00000080:00200000:1.0:1787827240.409207:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000b6e4e92e 17ffffc0001001 3:0 ffff9a788307ccb0 2 lru 00000080:00200000:1.0:1787827240.409211:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000033f168ef 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409213:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000cb7301cc 00000080:00200000:1.0:1787827240.409214:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:1.0:1787827240.409215:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409225:0:48370:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409228:0:48370:0:(rw26.c:933:ll_write_end()) page@000000006b34bd14[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 3] 00000080:00200000:1.0:1787827240.409230:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000001be6e032 17ffffc0001001 3:0 ffff9a788307cae0 3 lru 00000080:00200000:1.0:1787827240.409234:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000f5a3afe6 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409235:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000006b34bd14 00000080:00200000:1.0:1787827240.409237:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:1.0:1787827240.409238:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409247:0:48370:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409251:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000a523a98f[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 4] 00000080:00200000:1.0:1787827240.409254:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000008775ae29 17ffffc0001001 3:0 ffff9a788307c658 4 lru 00000080:00200000:1.0:1787827240.409260:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000007d57828a 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409262:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000a523a98f 00000080:00200000:1.0:1787827240.409263:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:1.0:1787827240.409266:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409282:0:48370:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409287:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000048ac0eb4[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 5] 00000080:00200000:1.0:1787827240.409290:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000524d1031 17ffffc0001001 3:0 ffff9a788307c9f8 5 lru 00000080:00200000:1.0:1787827240.409297:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000348f9056 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409300:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000048ac0eb4 00000080:00200000:1.0:1787827240.409301:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:1.0:1787827240.409303:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409321:0:48370:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409326:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000e39b596d[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 6] 00000080:00200000:1.0:1787827240.409329:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000d8a41f02 17ffffc0001001 3:0 ffff9a788307c910 6 lru 00000080:00200000:1.0:1787827240.409335:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000009dadeeb6 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409338:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000e39b596d 00000080:00200000:1.0:1787827240.409339:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:1.0:1787827240.409341:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409357:0:48370:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409362:0:48370:0:(rw26.c:933:ll_write_end()) page@000000004be048f0[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 7] 00000080:00200000:1.0:1787827240.409364:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000901cefbb 17ffffc0001001 3:0 ffff9a788307cd98 7 lru 00000080:00200000:1.0:1787827240.409368:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000924087fe 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409370:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000004be048f0 00000080:00200000:1.0:1787827240.409371:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:1.0:1787827240.409372:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409383:0:48370:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409386:0:48370:0:(rw26.c:933:ll_write_end()) page@000000003a4d39ef[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 8] 00000080:00200000:1.0:1787827240.409388:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000003753dc2b 17ffffc0001001 3:0 ffff9a788307c3a0 8 lru 00000080:00200000:1.0:1787827240.409394:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000004a7f8956 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409397:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000003a4d39ef 00000080:00200000:1.0:1787827240.409398:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:1.0:1787827240.409400:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409416:0:48370:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409421:0:48370:0:(rw26.c:933:ll_write_end()) page@000000000d30ba37[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 9] 00000080:00200000:1.0:1787827240.409424:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000008236d370 17ffffc0001001 3:0 ffff9a788307c0e8 9 lru 00000080:00200000:1.0:1787827240.409431:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000a7765cc0 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409434:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000000d30ba37 00000080:00200000:1.0:1787827240.409435:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:1.0:1787827240.409437:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409453:0:48370:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409459:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000097869436[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 10] 00000080:00200000:1.0:1787827240.409462:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000022580fbd 17ffffc0001001 3:0 ffff9a788307c2b8 10 lru 00000080:00200000:1.0:1787827240.409468:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000080effc6a 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409471:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000097869436 00000080:00200000:1.0:1787827240.409472:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:1.0:1787827240.409474:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409497:0:48370:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409501:0:48370:0:(rw26.c:933:ll_write_end()) page@000000004965e033[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 11] 00000080:00200000:1.0:1787827240.409503:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000fa413728 17ffffc0001021 2:0 ffff9a788307c1d0 11 lru 00000080:00200000:1.0:1787827240.409507:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000035180d4e 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409525:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000004965e033 00000080:00200000:1.0:1787827240.409526:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:1.0:1787827240.409528:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409541:0:48370:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409544:0:48370:0:(rw26.c:933:ll_write_end()) page@000000004fc0bd13[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 12] 00000080:00200000:1.0:1787827240.409547:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000cfc96f8e 17ffffc0001001 3:0 ffff9a788307c740 12 lru 00000080:00200000:1.0:1787827240.409551:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000082ec80f1 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409553:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000004fc0bd13 00000080:00200000:1.0:1787827240.409554:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:1.0:1787827240.409556:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409569:0:48370:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409572:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000839ecfc9[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 13] 00000080:00200000:1.0:1787827240.409574:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000005d2dd417 17ffffc0001001 3:0 ffff9a788307ce80 13 lru 00000080:00200000:1.0:1787827240.409578:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000f66c4b55 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409580:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000839ecfc9 00000080:00200000:1.0:1787827240.409581:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:1.0:1787827240.409582:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409591:0:48370:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409594:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000f1337c67[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 14] 00000080:00200000:1.0:1787827240.409596:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000039128e50 17ffffc0001001 3:0 ffff9a788307c488 14 lru 00000080:00200000:1.0:1787827240.409599:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000072750aaf 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409601:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000f1337c67 00000080:00200000:1.0:1787827240.409602:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:1.0:1787827240.409603:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409613:0:48370:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409617:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000eea10167[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 15] 00000080:00200000:1.0:1787827240.409620:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000061399ef3 17ffffc0001001 3:0 ffff9a788307cbc8 15 lru 00000080:00200000:1.0:1787827240.409624:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000006cbe66e 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409626:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000eea10167 00000080:00200000:1.0:1787827240.409627:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:1.0:1787827240.409628:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409638:0:48370:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409641:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000d256b620[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 16] 00000080:00200000:1.0:1787827240.409643:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000007cd00ebb 17ffffc0001001 3:0 ffff9a788307c570 16 lru 00000080:00200000:1.0:1787827240.409647:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000dd348f40 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409649:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000d256b620 00000080:00200000:1.0:1787827240.409650:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000080:00200000:1.0:1787827240.409652:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409669:0:48370:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409672:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000655882c9[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 17] 00000080:00200000:1.0:1787827240.409675:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000000644c7ad 17ffffc0001001 3:0 ffff9a78997300e8 17 lru 00000080:00200000:1.0:1787827240.409682:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000007652c971 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409685:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000655882c9 00000080:00200000:1.0:1787827240.409688:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:1.0:1787827240.409691:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409703:0:48370:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409707:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000010cc830b[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 18] 00000080:00200000:1.0:1787827240.409709:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000e47b672e 17ffffc0001001 3:0 ffff9a78997302b8 18 lru 00000080:00200000:1.0:1787827240.409713:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000075a05c41 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409715:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000010cc830b 00000080:00200000:1.0:1787827240.409716:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:1.0:1787827240.409717:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409726:0:48370:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409729:0:48370:0:(rw26.c:933:ll_write_end()) page@000000004ba03325[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 19] 00000080:00200000:1.0:1787827240.409731:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000964e7c24 17ffffc0001001 3:0 ffff9a78997301d0 19 lru 00000080:00200000:1.0:1787827240.409734:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000a09bf617 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409736:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000004ba03325 00000080:00200000:1.0:1787827240.409737:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:1.0:1787827240.409738:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409748:0:48370:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409750:0:48370:0:(rw26.c:933:ll_write_end()) page@000000001508bfc6[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 20] 00000080:00200000:1.0:1787827240.409752:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000006c12823d 17ffffc0001001 3:0 ffff9a7899730740 20 lru 00000080:00200000:1.0:1787827240.409756:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000086f8d2a0 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409757:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000001508bfc6 00000080:00200000:1.0:1787827240.409758:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:1.0:1787827240.409760:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409768:0:48370:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409771:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000620c940f[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 21] 00000080:00200000:1.0:1787827240.409773:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000993ffd4e 17ffffc0001001 3:0 ffff9a7899730e80 21 lru 00000080:00200000:1.0:1787827240.409776:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000004a910463 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409778:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000620c940f 00000080:00200000:1.0:1787827240.409779:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:1.0:1787827240.409780:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409788:0:48370:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409791:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000fb6d9de0[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 22] 00000080:00200000:1.0:1787827240.409793:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000002d6e9f03 17ffffc0001001 3:0 ffff9a7899730488 22 lru 00000080:00200000:1.0:1787827240.409797:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000000769a14d 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409798:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000fb6d9de0 00000080:00200000:1.0:1787827240.409799:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:1.0:1787827240.409800:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409808:0:48370:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409811:0:48370:0:(rw26.c:933:ll_write_end()) page@000000007484e094[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 23] 00000080:00200000:1.0:1787827240.409813:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000006b6f6e9f 17ffffc0001001 3:0 ffff9a7899730bc8 23 lru 00000080:00200000:1.0:1787827240.409816:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000b0542149 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409817:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000007484e094 00000080:00200000:1.0:1787827240.409818:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:1.0:1787827240.409819:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409827:0:48370:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409854:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000312469bd[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 24] 00000080:00200000:1.0:1787827240.409857:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000000853ebde 17ffffc0001001 3:0 ffff9a7899730570 24 lru 00000080:00200000:1.0:1787827240.409864:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000a55ac0b3 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409866:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000312469bd 00000080:00200000:1.0:1787827240.409867:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:1.0:1787827240.409870:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409889:0:48370:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409893:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000087c8bd31[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 25] 00000080:00200000:1.0:1787827240.409895:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000795d94b9 17ffffc0001001 3:0 ffff9a7899730828 25 lru 00000080:00200000:1.0:1787827240.409899:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000004d835dae 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409901:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000087c8bd31 00000080:00200000:1.0:1787827240.409902:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:1.0:1787827240.409904:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409922:0:48370:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409928:0:48370:0:(rw26.c:933:ll_write_end()) page@000000008229e572[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 26] 00000080:00200000:1.0:1787827240.409931:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000dbf5238b 17ffffc0001021 2:0 ffff9a7899730000 26 lru 00000080:00200000:1.0:1787827240.409938:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000007bfb870c 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409940:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000008229e572 00000080:00200000:1.0:1787827240.409942:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:1.0:1787827240.409944:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409962:0:48370:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409966:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000ed8ce5ca[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 27] 00000080:00200000:1.0:1787827240.409968:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000edbd10c7 17ffffc0001001 3:0 ffff9a7899730cb0 27 lru 00000080:00200000:1.0:1787827240.409972:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000f392f73e 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409973:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000ed8ce5ca 00000080:00200000:1.0:1787827240.409974:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:1.0:1787827240.409976:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.409988:0:48370:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:1.0:1787827240.409990:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000edb14f75[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 28] 00000080:00200000:1.0:1787827240.409992:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000054db3130 17ffffc0001001 3:0 ffff9a7899730ae0 28 lru 00000080:00200000:1.0:1787827240.409996:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000006a557d2d 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.409998:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000edb14f75 00000080:00200000:1.0:1787827240.409999:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:1.0:1787827240.410000:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410009:0:48370:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410011:0:48370:0:(rw26.c:933:ll_write_end()) page@000000002bf97483[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 29] 00000080:00200000:1.0:1787827240.410013:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000b1e759d8 17ffffc0001001 3:0 ffff9a7899730658 29 lru 00000080:00200000:1.0:1787827240.410017:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000718728b8 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410018:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000002bf97483 00000080:00200000:1.0:1787827240.410019:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:1.0:1787827240.410020:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410029:0:48370:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410032:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000099b65d22[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 30] 00000080:00200000:1.0:1787827240.410034:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000a6e31e42 17ffffc0001001 3:0 ffff9a78997309f8 30 lru 00000080:00200000:1.0:1787827240.410037:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000003d064498 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410039:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000099b65d22 00000080:00200000:1.0:1787827240.410041:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:1.0:1787827240.410042:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410050:0:48370:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410053:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000a79ec61f[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 31] 00000080:00200000:1.0:1787827240.410055:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000025f18dff 17ffffc0001001 3:0 ffff9a7899730910 31 lru 00000080:00200000:1.0:1787827240.410059:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000002d3c2627 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410061:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000a79ec61f 00000080:00200000:1.0:1787827240.410062:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:1.0:1787827240.410063:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410071:0:48370:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410074:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000b16c71f3[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 32] 00000080:00200000:1.0:1787827240.410075:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000ef020f2f 17ffffc0001001 3:0 ffff9a7899730d98 32 lru 00000080:00200000:1.0:1787827240.410079:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000ce7478d1 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410080:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000b16c71f3 00000080:00200000:1.0:1787827240.410081:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:1.0:1787827240.410082:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410090:0:48370:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410093:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000f3425a64[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 33] 00000080:00200000:1.0:1787827240.410094:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000013ce1c1d 17ffffc0001001 3:0 ffff9a78997303a0 33 lru 00000080:00200000:1.0:1787827240.410098:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000f1e8d7d1 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410099:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000f3425a64 00000080:00200000:1.0:1787827240.410099:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:1.0:1787827240.410101:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410117:0:48370:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410120:0:48370:0:(rw26.c:933:ll_write_end()) page@000000008a50efab[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 34] 00000080:00200000:1.0:1787827240.410122:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000037579656 17ffffc0001001 3:0 ffff9a7887c92ae0 34 lru 00000080:00200000:1.0:1787827240.410125:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000012d46857 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410126:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000008a50efab 00000080:00200000:1.0:1787827240.410127:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:1.0:1787827240.410128:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410137:0:48370:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410141:0:48370:0:(rw26.c:933:ll_write_end()) page@000000008d7150ef[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 35] 00000080:00200000:1.0:1787827240.410145:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000009b048978 17ffffc0001001 3:0 ffff9a7887c92658 35 lru 00000080:00200000:1.0:1787827240.410150:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000015042abf 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410153:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000008d7150ef 00000080:00200000:1.0:1787827240.410154:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:1.0:1787827240.410156:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410171:0:48370:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410176:0:48370:0:(rw26.c:933:ll_write_end()) page@000000001bab0b80[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 36] 00000080:00200000:1.0:1787827240.410179:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000e78a1787 17ffffc0001001 3:0 ffff9a7887c929f8 36 lru 00000080:00200000:1.0:1787827240.410186:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000e6011108 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410189:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000001bab0b80 00000080:00200000:1.0:1787827240.410190:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:1.0:1787827240.410192:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410206:0:48370:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410209:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000fa66ac86[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 37] 00000080:00200000:1.0:1787827240.410211:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000561a173f 17ffffc0001001 3:0 ffff9a7887c92910 37 lru 00000080:00200000:1.0:1787827240.410215:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000f452348b 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410217:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000fa66ac86 00000080:00200000:1.0:1787827240.410218:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:1.0:1787827240.410219:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410228:0:48370:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410230:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000ec1e32bd[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 38] 00000080:00200000:1.0:1787827240.410233:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000064949814 17ffffc0001001 3:0 ffff9a7887c92d98 38 lru 00000080:00200000:1.0:1787827240.410236:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000042479f2b 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410238:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000ec1e32bd 00000080:00200000:1.0:1787827240.410239:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:1.0:1787827240.410240:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410249:0:48370:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410253:0:48370:0:(rw26.c:933:ll_write_end()) page@000000004abc0ff8[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 39] 00000080:00200000:1.0:1787827240.410256:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000000c7d890f 17ffffc0001001 3:0 ffff9a7887c923a0 39 lru 00000080:00200000:1.0:1787827240.410262:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000c0d1bcbe 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410264:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000004abc0ff8 00000080:00200000:1.0:1787827240.410266:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:1.0:1787827240.410268:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410285:0:48370:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410290:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000e69a8637[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 40] 00000080:00200000:1.0:1787827240.410294:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000000b45a923 17ffffc0001001 3:0 ffff9a7887c920e8 40 lru 00000080:00200000:1.0:1787827240.410300:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000395b3b72 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410302:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000e69a8637 00000080:00200000:1.0:1787827240.410304:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:1.0:1787827240.410307:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410328:0:48370:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410333:0:48370:0:(rw26.c:933:ll_write_end()) page@000000003c4123af[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 41] 00000080:00200000:1.0:1787827240.410336:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000003a8b30be 17ffffc0001021 2:0 ffff9a7887c922b8 41 lru 00000080:00200000:1.0:1787827240.410343:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000be2ba6e5 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410346:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000003c4123af 00000080:00200000:1.0:1787827240.410347:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:1.0:1787827240.410349:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410365:0:48370:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410369:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000066f19861[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 42] 00000080:00200000:1.0:1787827240.410371:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000035f9d1d9 17ffffc0001001 3:0 ffff9a7887c921d0 42 lru 00000080:00200000:1.0:1787827240.410375:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000032b2bc51 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410377:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000066f19861 00000080:00200000:1.0:1787827240.410378:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:1.0:1787827240.410379:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410389:0:48370:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410391:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000015676a76[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 43] 00000080:00200000:1.0:1787827240.410394:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000002a76bc5c 17ffffc0001001 3:0 ffff9a7887c92740 43 lru 00000080:00200000:1.0:1787827240.410397:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000db195ea5 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410399:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000015676a76 00000080:00200000:1.0:1787827240.410400:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:1.0:1787827240.410401:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410410:0:48370:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410412:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000cf2b63f7[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 44] 00000080:00200000:1.0:1787827240.410415:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000018e8990f 17ffffc0001001 3:0 ffff9a7887c92e80 44 lru 00000080:00200000:1.0:1787827240.410418:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000041b34e6e 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410419:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000cf2b63f7 00000080:00200000:1.0:1787827240.410420:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:1.0:1787827240.410421:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410429:0:48370:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410432:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000016512ad[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 45] 00000080:00200000:1.0:1787827240.410434:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000f861f6d2 17ffffc0001001 3:0 ffff9a7887c92488 45 lru 00000080:00200000:1.0:1787827240.410437:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000f47f89ff 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410439:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000016512ad 00000080:00200000:1.0:1787827240.410439:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:1.0:1787827240.410440:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410448:0:48370:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410451:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000b533a3a9[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 46] 00000080:00200000:1.0:1787827240.410452:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000fbb27a9f 17ffffc0001001 3:0 ffff9a7887c92bc8 46 lru 00000080:00200000:1.0:1787827240.410456:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000fc6e70b9 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410457:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000b533a3a9 00000080:00200000:1.0:1787827240.410458:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:1.0:1787827240.410459:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410468:0:48370:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410470:0:48370:0:(rw26.c:933:ll_write_end()) page@000000008da746fb[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 47] 00000080:00200000:1.0:1787827240.410473:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000079d7ae45 17ffffc0001001 3:0 ffff9a7887c92570 47 lru 00000080:00200000:1.0:1787827240.410476:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000064d2f622 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410478:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000008da746fb 00000080:00200000:1.0:1787827240.410478:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:1.0:1787827240.410479:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410489:0:48370:0:(rw26.c:875:ll_write_end()) pos 196608, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410491:0:48370:0:(rw26.c:933:ll_write_end()) page@000000009fd280ad[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 48] 00000080:00200000:1.0:1787827240.410493:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000006e0c34ab 17ffffc0001001 3:0 ffff9a7887c92828 48 lru 00000080:00200000:1.0:1787827240.410496:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000ee0aeff2 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410498:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000009fd280ad 00000080:00200000:1.0:1787827240.410499:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:1.0:1787827240.410500:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 49 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410523:0:48370:0:(rw26.c:875:ll_write_end()) pos 200704, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410527:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000080229a46[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 49] 00000080:00200000:1.0:1787827240.410529:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000cdc251d0 17ffffc0001001 3:0 ffff9a7887c92000 49 lru 00000080:00200000:1.0:1787827240.410533:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000c60642d9 49: 1< 0 - - > 2< 200704 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410535:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000080229a46 00000080:00200000:1.0:1787827240.410536:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 50. 00000080:00200000:1.0:1787827240.410538:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 50 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410549:0:48370:0:(rw26.c:875:ll_write_end()) pos 204800, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410551:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000f4702146[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 50] 00000080:00200000:1.0:1787827240.410553:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000061a2bda0 17ffffc0001001 3:0 ffff9a7887c92cb0 50 lru 00000080:00200000:1.0:1787827240.410557:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000038b8f98d 50: 1< 0 - - > 2< 204800 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410559:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000f4702146 00000080:00200000:1.0:1787827240.410560:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 51. 00000080:00200000:1.0:1787827240.410561:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 51 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410576:0:48370:0:(rw26.c:875:ll_write_end()) pos 208896, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410579:0:48370:0:(rw26.c:933:ll_write_end()) page@000000000533cda8[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 51] 00000080:00200000:1.0:1787827240.410581:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000001bf59d19 17ffffc0001001 3:0 ffff9a78878862b8 51 lru 00000080:00200000:1.0:1787827240.410584:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000e607c594 51: 1< 0 - - > 2< 208896 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410586:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000000533cda8 00000080:00200000:1.0:1787827240.410587:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 52. 00000080:00200000:1.0:1787827240.410588:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 52 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410597:0:48370:0:(rw26.c:875:ll_write_end()) pos 212992, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410600:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000e92d211b[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 52] 00000080:00200000:1.0:1787827240.410602:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000050e8371 17ffffc0001001 3:0 ffff9a78878861d0 52 lru 00000080:00200000:1.0:1787827240.410606:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000007faa05e2 52: 1< 0 - - > 2< 212992 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410608:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000e92d211b 00000080:00200000:1.0:1787827240.410608:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 53. 00000080:00200000:1.0:1787827240.410609:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 53 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410617:0:48370:0:(rw26.c:875:ll_write_end()) pos 217088, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410620:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000070b41353[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 53] 00000080:00200000:1.0:1787827240.410622:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000adfe21b5 17ffffc0001001 3:0 ffff9a7887886740 53 lru 00000080:00200000:1.0:1787827240.410626:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000762f3348 53: 1< 0 - - > 2< 217088 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410628:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000070b41353 00000080:00200000:1.0:1787827240.410629:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 54. 00000080:00200000:1.0:1787827240.410630:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 54 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410638:0:48370:0:(rw26.c:875:ll_write_end()) pos 221184, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410640:0:48370:0:(rw26.c:933:ll_write_end()) page@000000002433cce4[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 54] 00000080:00200000:1.0:1787827240.410642:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000041f34d0 17ffffc0001001 3:0 ffff9a7887886e80 54 lru 00000080:00200000:1.0:1787827240.410645:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000fd3e5437 54: 1< 0 - - > 2< 221184 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410647:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000002433cce4 00000080:00200000:1.0:1787827240.410648:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 55. 00000080:00200000:1.0:1787827240.410649:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 55 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410659:0:48370:0:(rw26.c:875:ll_write_end()) pos 225280, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410662:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000059ec7c53[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 55] 00000080:00200000:1.0:1787827240.410664:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000776d2f28 17ffffc0001001 3:0 ffff9a7887886488 55 lru 00000080:00200000:1.0:1787827240.410668:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000138dfd4d 55: 1< 0 - - > 2< 225280 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410669:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000059ec7c53 00000080:00200000:1.0:1787827240.410670:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 56. 00000080:00200000:1.0:1787827240.410671:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 56 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410682:0:48370:0:(rw26.c:875:ll_write_end()) pos 229376, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410685:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000574e92c9[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 56] 00000080:00200000:1.0:1787827240.410686:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000002f8151d3 17ffffc0001021 2:0 ffff9a7887886bc8 56 lru 00000080:00200000:1.0:1787827240.410690:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000a3d3de21 56: 1< 0 - - > 2< 229376 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410691:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000574e92c9 00000080:00200000:1.0:1787827240.410692:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 57. 00000080:00200000:1.0:1787827240.410693:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 57 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410702:0:48370:0:(rw26.c:875:ll_write_end()) pos 233472, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410705:0:48370:0:(rw26.c:933:ll_write_end()) page@000000006177badd[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 57] 00000080:00200000:1.0:1787827240.410707:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000072ee6ed9 17ffffc0001001 3:0 ffff9a7887886570 57 lru 00000080:00200000:1.0:1787827240.410711:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000d5b458a3 57: 1< 0 - - > 2< 233472 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410712:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000006177badd 00000080:00200000:1.0:1787827240.410713:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 58. 00000080:00200000:1.0:1787827240.410715:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 58 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410723:0:48370:0:(rw26.c:875:ll_write_end()) pos 237568, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410726:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000644d52cf[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 58] 00000080:00200000:1.0:1787827240.410728:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000008d87fa7c 17ffffc0001001 3:0 ffff9a7887886828 58 lru 00000080:00200000:1.0:1787827240.410732:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000007bcd9238 58: 1< 0 - - > 2< 237568 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410733:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000644d52cf 00000080:00200000:1.0:1787827240.410734:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 59. 00000080:00200000:1.0:1787827240.410735:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 59 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410749:0:48370:0:(rw26.c:875:ll_write_end()) pos 241664, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410752:0:48370:0:(rw26.c:933:ll_write_end()) page@000000004364155e[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 59] 00000080:00200000:1.0:1787827240.410753:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000c1f23ad4 17ffffc0001001 3:0 ffff9a7887886000 59 lru 00000080:00200000:1.0:1787827240.410757:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000033ec985f 59: 1< 0 - - > 2< 241664 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410758:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000004364155e 00000080:00200000:1.0:1787827240.410759:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 60. 00000080:00200000:1.0:1787827240.410760:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 60 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410768:0:48370:0:(rw26.c:875:ll_write_end()) pos 245760, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410771:0:48370:0:(rw26.c:933:ll_write_end()) page@000000003d98dbee[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 60] 00000080:00200000:1.0:1787827240.410773:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000742d83ce 17ffffc0001001 3:0 ffff9a7887886cb0 60 lru 00000080:00200000:1.0:1787827240.410777:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000454cd53f 60: 1< 0 - - > 2< 245760 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410778:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000003d98dbee 00000080:00200000:1.0:1787827240.410779:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 61. 00000080:00200000:1.0:1787827240.410780:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 61 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410788:0:48370:0:(rw26.c:875:ll_write_end()) pos 249856, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410791:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000f26e1bc5[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 61] 00000080:00200000:1.0:1787827240.410792:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000001afdaa63 17ffffc0001001 3:0 ffff9a7887886ae0 61 lru 00000080:00200000:1.0:1787827240.410796:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000007b393b78 61: 1< 0 - - > 2< 249856 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410797:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000f26e1bc5 00000080:00200000:1.0:1787827240.410798:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 62. 00000080:00200000:1.0:1787827240.410799:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 62 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410808:0:48370:0:(rw26.c:875:ll_write_end()) pos 253952, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410811:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000a57f6cf3[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 62] 00000080:00200000:1.0:1787827240.410813:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000043af7cd7 17ffffc0001001 3:0 ffff9a7887886658 62 lru 00000080:00200000:1.0:1787827240.410817:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000006e62d8d 62: 1< 0 - - > 2< 253952 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410819:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000a57f6cf3 00000080:00200000:1.0:1787827240.410820:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 63. 00000080:00200000:1.0:1787827240.410821:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 63 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410847:0:48370:0:(rw26.c:875:ll_write_end()) pos 258048, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410853:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000e12a7095[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 63] 00000080:00200000:1.0:1787827240.410856:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000a462d622 17ffffc0001001 3:0 ffff9a78878869f8 63 lru 00000080:00200000:1.0:1787827240.410862:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000168d1211 63: 1< 0 - - > 2< 258048 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410864:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000e12a7095 00000080:00200000:1.0:1787827240.410866:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 64. 00000080:00200000:1.0:1787827240.410868:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 64 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410892:0:48370:0:(rw26.c:875:ll_write_end()) pos 262144, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410897:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000a0a2c0c3[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 64] 00000080:00200000:1.0:1787827240.410899:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000023a7040f 17ffffc0001001 3:0 ffff9a7887886910 64 lru 00000080:00200000:1.0:1787827240.410904:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000005d0f125c 64: 1< 0 - - > 2< 262144 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410906:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000a0a2c0c3 00000080:00200000:1.0:1787827240.410908:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 65. 00000080:00200000:1.0:1787827240.410910:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 65 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410927:0:48370:0:(rw26.c:875:ll_write_end()) pos 266240, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410932:0:48370:0:(rw26.c:933:ll_write_end()) page@000000002b5d2dd3[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 65] 00000080:00200000:1.0:1787827240.410935:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000e4f545b7 17ffffc0001001 3:0 ffff9a7887886d98 65 lru 00000080:00200000:1.0:1787827240.410941:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000051f83661 65: 1< 0 - - > 2< 266240 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410944:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000002b5d2dd3 00000080:00200000:1.0:1787827240.410945:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 66. 00000080:00200000:1.0:1787827240.410947:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 66 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410963:0:48370:0:(rw26.c:875:ll_write_end()) pos 270336, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410966:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000ad66d6d8[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 66] 00000080:00200000:1.0:1787827240.410969:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000bc66cbf5 17ffffc0001001 3:0 ffff9a78878863a0 66 lru 00000080:00200000:1.0:1787827240.410973:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000002c4749df 66: 1< 0 - - > 2< 270336 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410975:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000ad66d6d8 00000080:00200000:1.0:1787827240.410976:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 67. 00000080:00200000:1.0:1787827240.410977:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 67 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.410986:0:48370:0:(rw26.c:875:ll_write_end()) pos 274432, len 4096, copied 4096 00000080:00200000:1.0:1787827240.410989:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000089c2f69f[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 67] 00000080:00200000:1.0:1787827240.410991:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000006c1ecbd8 17ffffc0001001 3:0 ffff9a78878860e8 67 lru 00000080:00200000:1.0:1787827240.410995:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000095027555 67: 1< 0 - - > 2< 274432 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.410997:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000089c2f69f 00000080:00200000:1.0:1787827240.410997:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 68. 00000080:00200000:1.0:1787827240.410998:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 68 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411013:0:48370:0:(rw26.c:875:ll_write_end()) pos 278528, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411016:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000d6722c48[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 68] 00000080:00200000:1.0:1787827240.411018:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000002264446d 17ffffc0001001 3:0 ffff9a7887aef9f8 68 lru 00000080:00200000:1.0:1787827240.411021:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000e33baea8 68: 1< 0 - - > 2< 278528 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411022:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000d6722c48 00000080:00200000:1.0:1787827240.411023:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 69. 00000080:00200000:1.0:1787827240.411025:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 69 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411035:0:48370:0:(rw26.c:875:ll_write_end()) pos 282624, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411038:0:48370:0:(rw26.c:933:ll_write_end()) page@000000006605a59d[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 69] 00000080:00200000:1.0:1787827240.411040:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000008c789e54 17ffffc0001001 3:0 ffff9a7887aef910 69 lru 00000080:00200000:1.0:1787827240.411044:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000001fa7e3ea 69: 1< 0 - - > 2< 282624 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411045:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000006605a59d 00000080:00200000:1.0:1787827240.411046:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 70. 00000080:00200000:1.0:1787827240.411047:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 70 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411055:0:48370:0:(rw26.c:875:ll_write_end()) pos 286720, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411057:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000030e44a0f[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 70] 00000080:00200000:1.0:1787827240.411059:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000009cfdcc88 17ffffc0001001 3:0 ffff9a7887aefd98 70 lru 00000080:00200000:1.0:1787827240.411063:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000049d0472e 70: 1< 0 - - > 2< 286720 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411064:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000030e44a0f 00000080:00200000:1.0:1787827240.411064:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 71. 00000080:00200000:1.0:1787827240.411066:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 71 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411077:0:48370:0:(rw26.c:875:ll_write_end()) pos 290816, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411080:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000a36ddbf2[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 71] 00000080:00200000:1.0:1787827240.411082:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000494b808c 17ffffc0001021 2:0 ffff9a7887aef3a0 71 lru 00000080:00200000:1.0:1787827240.411086:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000b02fc0da 71: 1< 0 - - > 2< 290816 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411088:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000a36ddbf2 00000080:00200000:1.0:1787827240.411089:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 72. 00000080:00200000:1.0:1787827240.411090:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 72 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411098:0:48370:0:(rw26.c:875:ll_write_end()) pos 294912, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411101:0:48370:0:(rw26.c:933:ll_write_end()) page@000000005b3a3115[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 72] 00000080:00200000:1.0:1787827240.411103:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000098389a9a 17ffffc0001001 3:0 ffff9a7887aef0e8 72 lru 00000080:00200000:1.0:1787827240.411106:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000099cca30d 72: 1< 0 - - > 2< 294912 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411108:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000005b3a3115 00000080:00200000:1.0:1787827240.411109:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 73. 00000080:00200000:1.0:1787827240.411110:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 73 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411118:0:48370:0:(rw26.c:875:ll_write_end()) pos 299008, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411121:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000023cce558[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 73] 00000080:00200000:1.0:1787827240.411123:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000002225fa78 17ffffc0001001 3:0 ffff9a7887aef2b8 73 lru 00000080:00200000:1.0:1787827240.411127:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000703a26f6 73: 1< 0 - - > 2< 299008 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411128:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000023cce558 00000080:00200000:1.0:1787827240.411129:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 74. 00000080:00200000:1.0:1787827240.411130:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 74 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411139:0:48370:0:(rw26.c:875:ll_write_end()) pos 303104, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411143:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000049cbe924[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 74] 00000080:00200000:1.0:1787827240.411146:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @000000001611734b 17ffffc0001001 3:0 ffff9a7887aef1d0 74 lru 00000080:00200000:1.0:1787827240.411152:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000be936213 74: 1< 0 - - > 2< 303104 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411154:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000049cbe924 00000080:00200000:1.0:1787827240.411155:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 75. 00000080:00200000:1.0:1787827240.411157:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 75 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411172:0:48370:0:(rw26.c:875:ll_write_end()) pos 307200, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411177:0:48370:0:(rw26.c:933:ll_write_end()) page@000000006047e6d8[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 75] 00000080:00200000:1.0:1787827240.411181:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000410099ff 17ffffc0001001 3:0 ffff9a7887aef740 75 lru 00000080:00200000:1.0:1787827240.411188:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@0000000059f8e64f 75: 1< 0 - - > 2< 307200 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411190:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000006047e6d8 00000080:00200000:1.0:1787827240.411192:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 76. 00000080:00200000:1.0:1787827240.411194:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 76 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.411208:0:48370:0:(rw26.c:875:ll_write_end()) pos 311296, len 4096, copied 4096 00000080:00200000:1.0:1787827240.411212:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000001c1906d[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 76] 00000080:00200000:1.0:1787827240.411214:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000a594e18f 17ffffc0001001 3:0 ffff9a7887aefe80 76 lru 00000080:00200000:1.0:1787827240.411217:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000bf977c53 76: 1< 0 - - > 2< 311296 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411219:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000001c1906d 00000080:00200000:1.0:1787827240.411220:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 77. 00000080:00200000:1.0:1787827240.411221:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 77 of 0 to 2156 bytes 00000080:00200000:1.0:1787827240.411238:0:48370:0:(rw26.c:875:ll_write_end()) pos 315392, len 2156, copied 2156 00000080:00200000:1.0:1787827240.411241:0:48370:0:(rw26.c:933:ll_write_end()) page@0000000016c75349[3 000000000c166a9d 2 1 00000000f1b7f9f4 0 0 77] 00000080:00200000:1.0:1787827240.411243:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000207bb7a5 17ffffc0001001 3:0 ffff9a7887aef488 77 lru 00000080:00200000:1.0:1787827240.411246:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000001ff27847 77: 1< 0 - - > 2< 315392 0 0 0x0 0x0 | 0000000043fddcb7 0000000060a98b36 > 3< 0 0 > 4< 0 0 8 4382720 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.411248:0:48370:0:(rw26.c:933:ll_write_end()) end page@0000000016c75349 00000080:00200000:1.0:1787827240.411249:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 78. 00000080:00200000:1.0:1787827240.411254:0:48370:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 78, from 0, to 2156 prio 0 00000008:00010000:1.0:1787827240.411285:0:48370:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411307:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411318:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411329:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411339:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411356:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411367:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411375:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411383:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411388:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411393:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411398:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411403:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411408:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411414:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.411426:0:48370:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916fabb8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.411431:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411436:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411442:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411447:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411452:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411457:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411462:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411471:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411476:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411481:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411486:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411491:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411496:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411501:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411506:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.411530:0:48370:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916fabb8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.411534:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411540:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411546:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411551:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411556:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411561:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411566:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411571:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411575:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411585:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411591:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411595:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411600:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411605:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411610:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.411617:0:48370:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916fabb8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.411620:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411625:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411633:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411638:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411642:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411647:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411652:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411657:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411662:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411667:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411671:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411677:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411682:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411686:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411691:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.411698:0:48370:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916fabb8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.411701:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411706:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411711:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411716:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411724:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411729:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411733:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411738:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411743:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411748:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411753:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411757:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411762:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411768:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411773:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.411780:0:48370:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916fabb8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.411783:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411788:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.411793:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.411798:0:48370:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916fabb8, count 3, dirtied 3 00000080:00200000:1.0:1787827240.411801:0:48370:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 78 pages 317548 bytes, tot: 317548 00000080:00200000:1.0:1787827240.411803:0:48370:0:(vvp_io.c:1421:vvp_io_write_start()) test-framework.sh: write bytes 0, result: 317548 00000020:00010000:1.0:1787827240.411818:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787827240.411820:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.411823:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7886af01b0 (ffff9a7885afd488/1) at cl_lock_cancel():142 00010000:00010000:1.0:1787827240.411827:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 5/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.411856:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:1.0:1787827240.411871:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787827240.411874:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7886af01b0 (ffff9a7885afd7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787827240.411884:0:48370:0:(vvp_io.c:533:vvp_io_advance()) advancing 317548 bytes 00000080:00200000:1.0:1787827240.411893:0:48370:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 77] 00000080:00200000:1.0:1787827240.411901:0:48370:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4e:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787827240.411906:0:48370:0:(file.c:2240:ll_file_io_generic()) test-framework.sh: 2 io complete with rc: 0, result: 317548, restart: 0 00000080:00200000:1.0:1787827240.411914:0:48370:0:(file.c:1775:ll_merge_attr_nolock()) [0x200001b72:0x4e:0x0] updating i_size 317548 i_blocks 0 00000080:00200000:1.0:1787827240.411919:0:48370:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 317548 00000080:00200008:1.0:1787827240.411951:0:48370:0:(file.c:479:ll_file_release()) START file test-framework.sh:[0x200001b72:0x4e:0x0](ffff9a78916faa08), flags 100001 00000002:00100000:1.0:1787827240.412000:0:48370:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a67ecf00 x1874671476527360/t12884902241(12884902241) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787827240.412011:0:48370:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787827240.412015:0:48370:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ef0c0 x1874671476528000/t0(0) o35->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787827240.412031:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ef0c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528000:192.168.203.140@tcp:35:cp.0 00000100:00100000:1.0:1787827240.412079:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.413130:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ef0c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528000:192.168.203.140@tcp:35:cp.0 00000002:00100000:1.0:1787827240.413148:0:48370:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a67ecf00 x1874671476527360/t12884902241(12884902241) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787827240.413199:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.413216:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.413238:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.413268:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.413278:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.413285:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.413290:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.413298:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e03b58 00010000:00010000:1.0:1787827240.413302:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.413309:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.413318:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.413325:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.413330:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.413342:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.413345:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.413347:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=test-framework.sh, flags=8 00000080:00200008:1.0:1787827240.413354:0:48370:0:(file.c:6313:ll_getattr_dentry()) START file test-framework.sh:[0x200001b72:0x4e:0x0](ffff9a78916faa08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827240.413358:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200001b72:0x4e:0x0](ffff9a78916faa08),name=test-framework.sh 00000002:00010000:1.0:1787827240.413370:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200001b72:0x4e:0x0]) in obj [0x200001b72:0x2:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.413377:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827240.413384:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827240.413389:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200001b72:0x2:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.413392:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x4e:0x0](ffff9a78916faa08) 00000080:00010000:1.0:1787827240.413394:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x4e:0x0](ffff9a78916faa08) for lock 0xebdcd48a4298e434 00000080:00010000:1.0:1787827240.413397:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e434 from it ffffaa5487e03cb8 00010000:00010000:1.0:1787827240.413399:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827240.413404:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827240.413418:0:48370:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4e:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787827240.413426:0:48370:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200001b72:0x4e:0x0] 00000008:00010000:1.0:1787827240.413439:0:48370:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787827240.413442:0:48370:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7886af01b0, osc lock ffff9a789822a138, flags 0x400001002 00010000:00010000:1.0:1787827240.413454:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.413461:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787827240.413469:0:48370:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787827240.413472:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 6/0,1 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.413477:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787827240.413489:0:48370:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787827240.413493:0:48370:0:(file.c:1775:ll_merge_attr_nolock()) [0x200001b72:0x4e:0x0] updating i_size 317548 i_blocks 0 00000020:00010000:1.0:1787827240.413497:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787827240.413499:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.413501:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7886af01b0 (ffff9a7885afd488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.413504:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787827240.413505:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7886af01b0 (ffff9a7885afd7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787827240.413526:0:48370:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4e:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787827240.413568:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.413574:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.413580:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.413586:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.413591:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.413594:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.413596:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.413599:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e03b58 00010000:00010000:1.0:1787827240.413601:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.413607:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.413617:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.413620:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:1.0:1787827240.413622:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.413624:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=25 00000080:00200000:1.0:1787827240.413626:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.413635:0:48370:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=yaml.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=9 00000080:00200000:1.0:1787827240.413640:0:48370:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=yaml.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), intent=getattr 00000002:00010000:1.0:1787827240.413647:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: yaml.sh,[0x0:0x0:0x0]) in obj [0x200001b72:0x2:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787827240.413678:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.413684:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.413692:0:48370:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.413702:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ed2c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528128:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.413737:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.414920:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ed2c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528128:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.414929:0:48370:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 4/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414936:0:48370:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 4/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414941:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 4/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414946:0:48370:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 3/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414952:0:48370:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 4/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414956:0:48370:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78908c2000 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:1.0:1787827240.414960:0:48370:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 4/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414965:0:48370:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 4/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414969:0:48370:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 5/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414975:0:48370:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827240.414981:0:48370:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 3/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.414986:0:48370:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e449 lrc: 0/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.414996:0:48370:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a67ed2c0 x1874671476528128/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/584 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787827240.415006:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200001b72:0x2:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787827240.415019:0:48370:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 30 res 8589941618/2 (0 0) 00000080:00200000:1.0:1787827240.415044:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.415049:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.415054:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.415061:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.415066:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.415069:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.415071:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.415073:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e03b58 00010000:00010000:1.0:1787827240.415075:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.415080:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.415094:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.415097:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.415100:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.415102:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.415103:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.415109:0:48370:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=yaml.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=8 00000080:00200000:1.0:1787827240.415112:0:48370:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=yaml.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), intent=getattr 00000002:00010000:1.0:1787827240.415117:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: yaml.sh,[0x0:0x0:0x0]) in obj [0x200001b72:0x2:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787827240.415132:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.415137:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.415142:0:48370:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.415152:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ed2c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528256:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.415182:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.416297:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ed2c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528256:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.416309:0:48370:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 4/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416319:0:48370:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 4/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416327:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 4/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416335:0:48370:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 3/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416344:0:48370:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 4/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416350:0:48370:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78908c3600 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:1.0:1787827240.416352:0:48370:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 4/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416357:0:48370:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 4/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416362:0:48370:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 5/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416366:0:48370:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827240.416370:0:48370:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 3/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416375:0:48370:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e450 lrc: 0/0,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.416386:0:48370:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a67ed2c0 x1874671476528256/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/584 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787827240.416392:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200001b72:0x2:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787827240.416403:0:48370:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 30 res 8589941618/2 (0 0) 00000080:00200000:1.0:1787827240.416448:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.416455:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.416460:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.416468:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.416473:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.416476:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.416478:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.416481:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e03bf8 00010000:00010000:1.0:1787827240.416484:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.416489:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.416495:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.416499:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:1.0:1787827240.416502:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.416504:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=25 00000080:00200000:1.0:1787827240.416506:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.416534:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.416538:0:48370:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=yaml.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), file ffff9a788b0fa780, open_flags 80c1, mode 81a4 opened 0 00000080:00200000:1.0:1787827240.416544:0:48370:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=yaml.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), intent=open|creat 00000002:00010000:1.0:1787827240.416558:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: yaml.sh,[0x200001b72:0x4f:0x0]) in obj [0x200001b72:0x2:0x0], intent: open|creat flags 0100302 00010000:00010000:1.0:1787827240.416585:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.416590:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787827240.416598:0:48370:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787827240.416602:0:48370:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ed2c0 x1874671476528512/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787827240.416608:0:48370:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48370 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.416618:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ed2c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528512:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.416712:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.417987:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ed2c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528512:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.418001:0:48370:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 4/0,1 mode: --/CW res: [0x200001b72:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418019:0:48370:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418030:0:48370:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418040:0:48370:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418048:0:48370:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418055:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418063:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418071:0:48370:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 4/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.418082:0:48370:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a67ed2c0 x1874671476528512/t12884902243(12884902243) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787827240.418095:0:48370:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ed2c0 x1874671476528512/t12884902243(12884902243) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827240.418109:0:48370:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787827240.418119:0:48370:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418126:0:48370:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 4 res 8589941618/79 (0 0) 00000080:00200000:1.0:1787827240.418130:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200001b72:0x2:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787827240.418136:0:48370:0:(namei.c:1628:ll_create_it()) VFS Op:name=yaml.sh, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), intent=open|creat 00000080:00200000:1.0:1787827240.418165:0:48370:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x4f:0x0](ffff9a78916fcb08) 00000080:00200000:1.0:1787827240.418240:0:48370:0:(file.c:6854:ll_layout_conf()) [0x200001b72:0x4f:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787827240.418245:0:48370:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200001b72:0x4f:0x0](ffff9a78916fcb08) 00000080:00010000:1.0:1787827240.418248:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x4f:0x0](ffff9a78916fcb08) for lock 0xebdcd48a4298e457 00000080:00010000:1.0:1787827240.418254:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x4f:0x0](ffff9a78916fcb08) 00000080:00200008:1.0:1787827240.418261:0:48370:0:(file.c:1090:ll_file_open()) START file yaml.sh:[0x200001b72:0x4f:0x0](ffff9a78916fcb08), flags 100301 00000002:00100000:1.0:1787827240.418277:0:48370:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ed2c0 x1874671476528512/t12884902243(12884902243) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787827240.418308:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e457 from it ffff9a78908e1540 00010000:00010000:1.0:1787827240.418311:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418317:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.418329:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x4f:0x0](ffff9a78916fcb08), inode mode 81a4 mask 40 00000080:00200008:1.0:1787827240.418339:0:48370:0:(file.c:6313:ll_getattr_dentry()) START file yaml.sh:[0x200001b72:0x4f:0x0](ffff9a78916fcb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827240.418343:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200001b72:0x4f:0x0](ffff9a78916fcb08),name=yaml.sh 00000002:00010000:1.0:1787827240.418351:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200001b72:0x4f:0x0]) in obj [0x200001b72:0x2:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.418357:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.418368:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.418373:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200001b72:0x2:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.418375:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x4f:0x0](ffff9a78916fcb08) 00000080:00010000:1.0:1787827240.418377:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x4f:0x0](ffff9a78916fcb08) for lock 0xebdcd48a4298e457 00000080:00010000:1.0:1787827240.418380:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e457 from it ffffaa5487e03cd8 00010000:00010000:1.0:1787827240.418382:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.418387:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827240.418397:0:48370:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4f:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787827240.418405:0:48370:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200001b72:0x4f:0x0] 00000008:00010000:1.0:1787827240.418416:0:48370:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787827240.418419:0:48370:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7886af0330, osc lock ffff9a789822a138, flags 0x400001002 00010000:00010000:1.0:1787827240.418443:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000007c30655/0xebdcd48a4298e45e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.418450:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000007c30655/0xebdcd48a4298e45e lrc: 3/1,0 mode: --/PR res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.418474:0:48370:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:1.0:1787827240.419439:0:48370:0:(file.c:1775:ll_merge_attr_nolock()) [0x200001b72:0x4f:0x0] updating i_size 0 i_blocks 0 00000020:00010000:1.0:1787827240.419444:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787827240.419447:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.419450:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7886af0330 (ffff9a7885afd488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.419454:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787827240.419457:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7886af0330 (ffff9a7885afd7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787827240.419466:0:48370:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4f:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:1.0:1787827240.419496:0:48370:0:(file.c:2652:do_file_write_iter()) START file yaml.sh:[0x200001b72:0x4f:0x0], ppos: 0, count: 4512 00000080:00200000:1.0:1787827240.419503:0:48370:0:(file.c:2090:ll_file_io_generic()) yaml.sh: write ppos: 0, bytes: 4512 00000080:00200000:1.0:1787827240.419522:0:48370:0:(file.c:1848:ll_io_set_mirror()) yaml.sh: desiginated mirror: 0 00000020:00200000:1.0:1787827240.419528:0:48370:0:(cl_io.c:218:cl_io_rw_init()) header@000000001a26bcce[0x110, 1, [0x200001b72:0x4f:0x0] hash] 00000020:00200000:1.0:1787827240.419530:0:48370:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 4512) 0 0 00000080:00200000:1.0:1787827240.419533:0:48370:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4f:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.419540:0:48370:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 1] 00020000:00200000:1.0:1787827240.419548:0:48370:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 4512) 4512, 4512 00020000:00200000:1.0:1787827240.419551:0:48370:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787827240.419563:0:48370:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 1] 00000008:00010000:1.0:1787827240.419572:0:48370:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:1.0:1787827240.419575:0:48370:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7886af0330, osc lock ffff9a789822a138, flags 0x0 00010000:00010000:1.0:1787827240.419582:0:48370:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419265/136 (0 8191) 00010000:00010000:1.0:1787827240.419599:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.419606:0:48370:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.419612:0:48370:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.419623:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67eda40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528768:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.419653:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393240 going to sleep for 11 seconds 00010000:00010000:1.0:1787827240.420581:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.420590:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000007c30655/0xebdcd48a4298e45e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dab3d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.420598:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78908c3c00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.420603:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000007c30655/0xebdcd48a4298e45e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dab3d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.420617:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.420625:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.420651:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000007c30655/0xebdcd48a4298e45e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dab3d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.420657:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.420675:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000007c30655/0xebdcd48a4298e45e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dab3d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.420682:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000007c30655/0xebdcd48a4298e45e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dab3d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.420695:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.420699:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.420704:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.421232:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67eda40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476528768:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.421245:0:48370:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.421254:0:48370:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.421262:0:48370:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.421786:0:48370:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.421805:0:48370:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.421815:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.421823:0:48370:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,2 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.421855:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.421866:0:48370:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.421889:0:48370:0:(vvp_io.c:1339:vvp_io_write_start()) yaml.sh: write [0, 4512) append 0 append lockpos 0 00000080:00200000:1.0:1787827240.421905:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.421942:0:48370:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:1.0:1787827240.421956:0:48370:0:(rw26.c:933:ll_write_end()) page@000000005493f80f[3 00000000b506518c 2 1 00000000f1b7f9f4 0 0 0] 00000080:00200000:1.0:1787827240.421960:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @0000000075a69971 17ffffc0001001 3:0 ffff9a7887aefbc8 0 lru 00000080:00200000:1.0:1787827240.421967:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@00000000590f445f 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 00000000e39b2a33 > 3< 0 0 > 4< 0 0 8 5984256 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.421969:0:48370:0:(rw26.c:933:ll_write_end()) end page@000000005493f80f 00000080:00200000:1.0:1787827240.421970:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:1.0:1787827240.421973:0:48370:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 416 bytes 00000080:00200000:1.0:1787827240.422001:0:48370:0:(rw26.c:875:ll_write_end()) pos 4096, len 416, copied 416 00000080:00200000:1.0:1787827240.422007:0:48370:0:(rw26.c:933:ll_write_end()) page@00000000c220fa46[3 00000000b506518c 2 1 00000000f1b7f9f4 0 0 1] 00000080:00200000:1.0:1787827240.422011:0:48370:0:(rw26.c:933:ll_write_end()) vmpage @00000000f24aca9b 17ffffc0001001 3:0 ffff9a7887aef570 1 lru 00000080:00200000:1.0:1787827240.422017:0:48370:0:(rw26.c:933:ll_write_end()) osc-page@000000003d4cb15d 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000023a82262 00000000e39b2a33 > 3< 0 0 > 4< 0 0 8 5984256 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.422020:0:48370:0:(rw26.c:933:ll_write_end()) end page@00000000c220fa46 00000080:00200000:1.0:1787827240.422021:0:48370:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:1.0:1787827240.422025:0:48370:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 2, from 0, to 416 prio 0 00000008:00010000:1.0:1787827240.422053:0:48370:0:(osc_cache.c:858:osc_extent_find()) ### extent: 0000000068faf82e ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.422076:0:48370:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 0000000068faf82e ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.422091:0:48370:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916fccb8, count 2, dirtied 2 00000080:00200000:1.0:1787827240.422096:0:48370:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 2 pages 4512 bytes, tot: 4512 00000080:00200000:1.0:1787827240.422099:0:48370:0:(vvp_io.c:1421:vvp_io_write_start()) yaml.sh: write bytes 0, result: 4512 00000020:00010000:1.0:1787827240.422114:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787827240.422117:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.422121:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7886af0330 (ffff9a7885afd488/1) at cl_lock_cancel():142 00010000:00010000:1.0:1787827240.422126:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.422136:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:1.0:1787827240.422154:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787827240.422157:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7886af0330 (ffff9a7885afd7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787827240.422164:0:48370:0:(vvp_io.c:533:vvp_io_advance()) advancing 4512 bytes 00000080:00200000:1.0:1787827240.422171:0:48370:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 1] 00000080:00200000:1.0:1787827240.422177:0:48370:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4f:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787827240.422181:0:48370:0:(file.c:2240:ll_file_io_generic()) yaml.sh: 2 io complete with rc: 0, result: 4512, restart: 0 00000080:00200000:1.0:1787827240.422188:0:48370:0:(file.c:1775:ll_merge_attr_nolock()) [0x200001b72:0x4f:0x0] updating i_size 4512 i_blocks 0 00000080:00200000:1.0:1787827240.422192:0:48370:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 4512 00000080:00200008:1.0:1787827240.422213:0:48370:0:(file.c:479:ll_file_release()) START file yaml.sh:[0x200001b72:0x4f:0x0](ffff9a78916fcb08), flags 100001 00000002:00100000:1.0:1787827240.422258:0:48370:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a67ed2c0 x1874671476528512/t12884902243(12884902243) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787827240.422270:0:48370:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787827240.422276:0:48370:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67eda40 x1874671476529152/t0(0) o35->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787827240.422293:0:48370:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67eda40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476529152:192.168.203.140@tcp:35:cp.0 00000100:00100000:1.0:1787827240.422326:0:48370:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393240 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.423116:0:48370:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67eda40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48370:1874671476529152:192.168.203.140@tcp:35:cp.0 00000002:00100000:1.0:1787827240.423132:0:48370:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a67ed2c0 x1874671476528512/t12884902243(12884902243) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787827240.423170:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.423180:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.423189:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.423201:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.423206:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.423210:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.423212:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.423216:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e03b58 00010000:00010000:1.0:1787827240.423219:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.423224:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.423231:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.423235:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.423239:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.423241:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.423243:0:48370:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.423245:0:48370:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=yaml.sh, flags=8 00000080:00200008:1.0:1787827240.423250:0:48370:0:(file.c:6313:ll_getattr_dentry()) START file yaml.sh:[0x200001b72:0x4f:0x0](ffff9a78916fcb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827240.423254:0:48370:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200001b72:0x4f:0x0](ffff9a78916fcb08),name=yaml.sh 00000002:00010000:1.0:1787827240.423264:0:48370:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200001b72:0x4f:0x0]) in obj [0x200001b72:0x2:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.423271:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827240.423281:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827240.423290:0:48370:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200001b72:0x2:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.423294:0:48370:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x4f:0x0](ffff9a78916fcb08) 00000080:00010000:1.0:1787827240.423298:0:48370:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x4f:0x0](ffff9a78916fcb08) for lock 0xebdcd48a4298e457 00000080:00010000:1.0:1787827240.423302:0:48370:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e457 from it ffffaa5487e03cb8 00010000:00010000:1.0:1787827240.423305:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827240.423314:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827240.423325:0:48370:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4f:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:1.0:1787827240.423330:0:48370:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200001b72:0x4f:0x0] 00000008:00010000:1.0:1787827240.423337:0:48370:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1787827240.423340:0:48370:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7886af0330, osc lock ffff9a789822a138, flags 0x400001002 00010000:00010000:1.0:1787827240.423348:0:48370:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.423355:0:48370:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:1.0:1787827240.423362:0:48370:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787827240.423365:0:48370:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 6/0,1 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.423370:0:48370:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000dcb0a1bb/0xebdcd48a4298e465 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dab44 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:1.0:1787827240.423381:0:48370:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:1.0:1787827240.423384:0:48370:0:(file.c:1775:ll_merge_attr_nolock()) [0x200001b72:0x4f:0x0] updating i_size 4512 i_blocks 0 00000020:00010000:1.0:1787827240.423387:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787827240.423389:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.423391:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7886af0330 (ffff9a7885afd488/1) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.423393:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877813c0 (ffff9a7885afd7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787827240.423395:0:48370:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7886af0330 (ffff9a7885afd7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787827240.423400:0:48370:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4f:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787827240.428771:0:48372:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.428793:0:48372:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.428809:0:48372:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.428822:0:48372:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.428853:0:48372:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.428862:0:48372:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.428867:0:48372:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.428876:0:48372:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa54816c7e18 00010000:00010000:1.0:1787827240.428881:0:48372:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.428892:0:48372:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.428915:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 101 00000080:00200000:1.0:1787827240.428927:0:48372:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.428934:0:48372:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.428942:0:48372:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.428952:0:48372:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.428961:0:48372:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.428966:0:48372:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.428969:0:48372:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.428973:0:48372:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa54816c7be0 00010000:00010000:1.0:1787827240.428975:0:48372:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.428981:0:48372:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.428987:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.428992:0:48372:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=1536 00000080:00200000:1.0:1787827240.429000:0:48372:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.429005:0:48372:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.429009:0:48372:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.429015:0:48372:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.429020:0:48372:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.429023:0:48372:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.429025:0:48372:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.429027:0:48372:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa54816c7c18 00010000:00010000:1.0:1787827240.429029:0:48372:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.429034:0:48372:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.429039:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.429042:0:48372:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=3 00000080:00200000:1.0:1787827240.429044:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 101 00000080:00200000:1.0:1787827240.429048:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.429051:0:48372:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=1536 00000080:00200000:1.0:1787827240.429055:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.429057:0:48372:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=3 00000080:00200000:1.0:1787827240.429059:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 101 00000080:00200000:1.0:1787827240.429063:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.429069:0:48372:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d_f6.sanity-scrub, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.429072:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.429074:0:48372:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.429079:0:48372:0:(namei.c:2239:do_mkdir()) VFS Op:name=d_f6.sanity-scrub, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) 00000020:00100000:1.0:1787827240.429132:0:48372:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.429142:0:48372:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ec000 x1874671476529408/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 568/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.429165:0:48372:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ec000 pname:cluuid:pid:xid:nid:opc:job mkdir:97196825-2cb9-49ba-a47d-1455a997ce3a:48372:1874671476529408:192.168.203.140@tcp:36:mkdir.0 00000100:00100000:1.0:1787827240.429240:0:48372:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393240 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.431015:0:48372:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ec000 pname:cluuid:pid:xid:nid:opc:job mkdir:97196825-2cb9-49ba-a47d-1455a997ce3a:48372:1874671476529408:192.168.203.140@tcp:36:mkdir.0 00000080:00200000:1.0:1787827240.431056:0:48372:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x50:0x0](ffff9a78916fba88) 00000080:00200000:1.0:1787827240.431064:0:48372:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200001b72:0x50:0x0], updating i_size 4096 00000080:00200000:1.0:1787827240.434714:0:48373:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.434738:0:48373:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.434759:0:48373:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.434777:0:48373:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.434788:0:48373:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.434795:0:48373:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.434799:0:48373:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.434808:0:48373:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e53bf8 00010000:00010000:1.0:1787827240.434812:0:48373:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.434822:0:48373:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.435209:0:48373:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.435217:0:48373:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:1.0:1787827240.435223:0:48373:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.435228:0:48373:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=25 00000080:00200000:1.0:1787827240.435232:0:48373:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.435242:0:48373:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d_f6.sanity-scrub, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=25 00000080:00200000:1.0:1787827240.435246:0:48373:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d_f6.sanity-scrub, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), intent=0 00000002:00010000:1.0:1787827240.435261:0:48373:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d_f6.sanity-scrub,[0x0:0x0:0x0]) in obj [0x200001b72:0x2:0x0], intent: lookup flags 0 00010000:00010000:1.0:1787827240.435311:0:48373:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.435321:0:48373:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.435333:0:48373:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.435358:0:48373:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ec780 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476529536:192.168.203.140@tcp:101:touch.0 00000100:00100000:1.0:1787827240.435444:0:48373:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393240 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.436884:0:48373:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ec780 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476529536:192.168.203.140@tcp:101:touch.0 00010000:00010000:1.0:1787827240.436901:0:48373:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 4/1,0 mode: --/CR res: [0x200001b72:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436918:0:48373:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 4/1,0 mode: --/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436925:0:48373:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 4/1,0 mode: --/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436935:0:48373:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 4/1,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436949:0:48373:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 4/1,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436958:0:48373:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 5/2,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436964:0:48373:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 5/2,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436968:0:48373:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 4/1,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.436976:0:48373:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff9a78a67ec780 x1874671476529536/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827240.436986:0:48373:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.436994:0:48373:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 4 res 8589941618/80 (0 0) 00000080:00200000:1.0:1787827240.436997:0:48373:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200001b72:0x2:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787827240.437014:0:48373:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200001b72:0x50:0x0], updating i_size 4096 00000080:00200000:1.0:1787827240.437017:0:48373:0:(namei.c:147:ll_iget()) got inode: [0x200001b72:0x50:0x0](ffff9a78916fba88): rc = 0 00000080:00010000:1.0:1787827240.437022:0:48373:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x50:0x0](ffff9a78916fba88) for lock 0xebdcd48a4298e46c 00000080:00010000:1.0:1787827240.437033:0:48373:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x50:0x0](ffff9a78916fba88) 00000080:00010000:1.0:1787827240.437035:0:48373:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e46c from it ffffaa5487e53ac8 00010000:00010000:1.0:1787827240.437037:0:48373:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.437048:0:48373:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.437066:0:48373:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x50:0x0](ffff9a78916fba88), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.437074:0:48373:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x50:0x0](ffff9a78916fba88), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.437077:0:48373:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f1, dir=[0x200001b72:0x50:0x0](ffff9a78916fba88), file ffff9a7887f09500, open_flags 8941, mode 81b6 opened 0 00000080:00200000:1.0:1787827240.437083:0:48373:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f1, dir=[0x200001b72:0x50:0x0](ffff9a78916fba88), intent=open|creat 00000002:00010000:1.0:1787827240.437098:0:48373:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f1,[0x200001b72:0x51:0x0]) in obj [0x200001b72:0x50:0x0], intent: open|creat flags 0104502 00010000:00010000:1.0:1787827240.437123:0:48373:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.437129:0:48373:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787827240.437147:0:48373:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787827240.437155:0:48373:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ec780 x1874671476529792/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787827240.437164:0:48373:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.437180:0:48373:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ec780 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476529792:192.168.203.140@tcp:101:touch.0 00000100:00100000:1.0:1787827240.437290:0:48373:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393240 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.438697:0:48373:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ec780 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476529792:192.168.203.140@tcp:101:touch.0 00010000:00010000:1.0:1787827240.438709:0:48373:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 4/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438724:0:48373:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438729:0:48373:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438736:0:48373:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438742:0:48373:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438748:0:48373:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438753:0:48373:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438758:0:48373:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 4/1,0 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.438765:0:48373:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a67ec780 x1874671476529792/t12884902246(12884902246) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787827240.438774:0:48373:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ec780 x1874671476529792/t12884902246(12884902246) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827240.438783:0:48373:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787827240.438790:0:48373:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.438796:0:48373:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 4 res 8589941618/81 (0 0) 00000080:00200000:1.0:1787827240.438800:0:48373:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200001b72:0x50:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787827240.438806:0:48373:0:(namei.c:1628:ll_create_it()) VFS Op:name=f1, dir=[0x200001b72:0x50:0x0](ffff9a78916fba88), intent=open|creat 00000080:00200000:1.0:1787827240.438826:0:48373:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x51:0x0](ffff9a78916f80c8) 00000080:00200000:1.0:1787827240.439048:0:48373:0:(file.c:6854:ll_layout_conf()) [0x200001b72:0x51:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787827240.439052:0:48373:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200001b72:0x51:0x0](ffff9a78916f80c8) 00000080:00010000:1.0:1787827240.439055:0:48373:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x51:0x0](ffff9a78916f80c8) for lock 0xebdcd48a4298e473 00000080:00010000:1.0:1787827240.439061:0:48373:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x51:0x0](ffff9a78916f80c8) 00000080:00200008:1.0:1787827240.439070:0:48373:0:(file.c:1090:ll_file_open()) START file f1:[0x200001b72:0x51:0x0](ffff9a78916f80c8), flags 104501 00000002:00100000:1.0:1787827240.439086:0:48373:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ec780 x1874671476529792/t12884902246(12884902246) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787827240.439115:0:48373:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e473 from it ffff9a78908e1540 00010000:00010000:1.0:1787827240.439118:0:48373:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.439125:0:48373:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.439138:0:48373:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x51:0x0](ffff9a78916f80c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1787827240.439166:0:48373:0:(llite_lib.c:2427:ll_setattr_raw()) START file f1:[0x200001b72:0x51:0x0](ffff9a78916f80c8) current size 0, valid attrs 20070, mode 9a78, uid -2015019440, gid -21932, new size 18446744071879195159, atime 1787827240.000000000, mtime 1787827240.000000000, ctime 1787827240.000000000 00010000:00010000:1.0:1787827240.439185:0:48373:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.439203:0:48373:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x51:0x0]: losing layout lock 00010000:00010000:1.0:1787827240.439226:0:48373:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.439234:0:48373:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.439237:0:48373:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3600/0xebdcd48a4298e473 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dab91 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787827240.439256:0:48373:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.439263:0:48373:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ee1c0 x1874671476530048/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.439281:0:48373:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ee1c0 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476530048:192.168.203.140@tcp:36:touch.0 00000100:00100000:1.0:1787827240.439322:0:48373:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.440294:0:48373:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ee1c0 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476530048:192.168.203.140@tcp:36:touch.0 00000080:00200000:1.0:1787827240.440317:0:48373:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x51:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00010000:1.0:1787827240.440329:0:48373:0:(file.c:7076:ll_layout_intent()) ### lustre: requeue layout lock for file [0x200001b72:0x51:0x0](ffff9a78916f80c8) 00000002:00010000:1.0:1787827240.440334:0:48373:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200001b72:0x51:0x0]) in obj [0x200001b72:0x51:0x0], intent: layout flags 0 00010000:00010000:1.0:1787827240.440355:0:48373:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.440361:0:48373:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.440367:0:48373:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: --/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 48373 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.440375:0:48373:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67eed00 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476530176:192.168.203.140@tcp:101:touch.0 00000100:00100000:1.0:1787827240.440404:0:48373:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.441176:0:48373:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67eed00 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476530176:192.168.203.140@tcp:101:touch.0 00010000:00010000:1.0:1787827240.441188:0:48373:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 4/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.441196:0:48373:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 4/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.441207:0:48373:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff9a78a67eed00 x1874671476530176/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 376/400 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827240.441217:0:48373:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787827240.441228:0:48373:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.441238:0:48373:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 16 res 8589941618/81 (0 0) 00000080:00200000:1.0:1787827240.441242:0:48373:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x200001b72:0x51:0x0] suppgids 0 0: rc 0 00000080:00010000:1.0:1787827240.441255:0:48373:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x51:0x0](ffff9a78916f80c8) for lock 0xebdcd48a4298e47a 00000080:00010000:1.0:1787827240.441260:0:48373:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e47a from it ffffaa5487e53a78 00010000:00010000:1.0:1787827240.441264:0:48373:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.441274:0:48373:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 2/0,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.441288:0:48373:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 2/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.441295:0:48373:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 2/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:1.0:1787827240.441302:0:48373:0:(file.c:6959:ll_layout_lock_set()) ### file [0x200001b72:0x51:0x0](00000000ece9d425) being reconfigured ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.441322:0:48373:0:(file.c:6854:ll_layout_conf()) [0x200001b72:0x51:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:1.0:1787827240.441325:0:48373:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 3/1,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.441330:0:48373:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 2/0,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00020000:00200000:1.0:1787827240.441350:0:48373:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787827240.442268:0:48373:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x51:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:1.0:1787827240.442278:0:48373:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f1:[0x200001b72:0x51:0x0](ffff9a78916f80c8) current size 0, valid attrs 100201f0, mode 9a78, uid -2015019440, gid -21932, new size 18446744071879195159, atime 1787827240.000000000, mtime 1787827240.000000000, ctime 1787827240.000000000, rc 0 00000080:00200008:1.0:1787827240.442295:0:48373:0:(file.c:479:ll_file_release()) START file f1:[0x200001b72:0x51:0x0](ffff9a78916f80c8), flags 104001 00000002:00100000:1.0:1787827240.442323:0:48373:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a67ec780 x1874671476529792/t12884902246(12884902246) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787827240.442331:0:48373:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787827240.442334:0:48373:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ed680 x1874671476530560/t0(0) o35->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787827240.442344:0:48373:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ed680 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476530560:192.168.203.140@tcp:35:touch.0 00000100:00100000:1.0:1787827240.442373:0:48373:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.443120:0:48373:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ed680 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48373:1874671476530560:192.168.203.140@tcp:35:touch.0 00000002:00100000:1.0:1787827240.443133:0:48373:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a67ec780 x1874671476529792/t12884902246(12884902246) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787827240.446579:0:48374:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.446596:0:48374:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.446611:0:48374:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.446624:0:48374:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.446630:0:48374:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.446635:0:48374:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.446638:0:48374:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.446645:0:48374:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e0bbf8 00010000:00010000:1.0:1787827240.446648:0:48374:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.446655:0:48374:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.446666:0:48374:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.446672:0:48374:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:1.0:1787827240.446677:0:48374:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.446679:0:48374:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=25 00000080:00200000:1.0:1787827240.446682:0:48374:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.446685:0:48374:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d_f6.sanity-scrub, flags=25 00000080:00200000:1.0:1787827240.446690:0:48374:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x50:0x0](ffff9a78916fba88), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.446697:0:48374:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x50:0x0](ffff9a78916fba88), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.446701:0:48374:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f2, dir=[0x200001b72:0x50:0x0](ffff9a78916fba88), file ffff9a78938f6300, open_flags 8241, mode 81b6 opened 0 00000080:00200000:1.0:1787827240.446708:0:48374:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f2, dir=[0x200001b72:0x50:0x0](ffff9a78916fba88), intent=open|creat 00000002:00010000:1.0:1787827240.446726:0:48374:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f2,[0x200001b72:0x52:0x0]) in obj [0x200001b72:0x50:0x0], intent: open|creat flags 0101102 00010000:00010000:1.0:1787827240.446766:0:48374:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48374 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.446772:0:48374:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48374 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787827240.446785:0:48374:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787827240.446795:0:48374:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ef840 x1874671476530816/t0(0) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787827240.446806:0:48374:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 3/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48374 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.446825:0:48374:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ef840 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48374:1874671476530816:192.168.203.140@tcp:101:dd.0 00000100:00100000:1.0:1787827240.446969:0:48374:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7891393f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.448539:0:48374:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ef840 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48374:1874671476530816:192.168.203.140@tcp:101:dd.0 00010000:00010000:1.0:1787827240.448558:0:48374:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 4/0,1 mode: --/CW res: [0x200001b72:0x50:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448578:0:48374:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448589:0:48374:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 4/0,1 mode: --/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448610:0:48374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448623:0:48374:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 4/0,1 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448636:0:48374:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448644:0:48374:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 5/1,1 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448652:0:48374:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 4/1,0 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787827240.448664:0:48374:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a67ef840 x1874671476530816/t12884902249(12884902249) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787827240.448679:0:48374:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ef840 x1874671476530816/t12884902249(12884902249) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787827240.448693:0:48374:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787827240.448707:0:48374:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448718:0:48374:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a788724f400 type 13 mode 4 res 8589941618/82 (0 0) 00000080:00200000:1.0:1787827240.448724:0:48374:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200001b72:0x50:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787827240.448735:0:48374:0:(namei.c:1628:ll_create_it()) VFS Op:name=f2, dir=[0x200001b72:0x50:0x0](ffff9a78916fba88), intent=open|creat 00000080:00200000:1.0:1787827240.448769:0:48374:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x52:0x0](ffff9a78916f9148) 00000080:00200000:1.0:1787827240.448875:0:48374:0:(file.c:6854:ll_layout_conf()) [0x200001b72:0x52:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787827240.448882:0:48374:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200001b72:0x52:0x0](ffff9a78916f9148) 00000080:00010000:1.0:1787827240.448885:0:48374:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200001b72:0x52:0x0](ffff9a78916f9148) for lock 0xebdcd48a4298e481 00000080:00010000:1.0:1787827240.448895:0:48374:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200001b72:0x52:0x0](ffff9a78916f9148) 00000080:00200008:1.0:1787827240.448905:0:48374:0:(file.c:1090:ll_file_open()) START file f2:[0x200001b72:0x52:0x0](ffff9a78916f9148), flags 101101 00000002:00100000:1.0:1787827240.448925:0:48374:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a67ef840 x1874671476530816/t12884902249(12884902249) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787827240.448958:0:48374:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298e481 from it ffff9a78908e1c00 00010000:00010000:1.0:1787827240.448964:0:48374:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 3/1,0 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.448974:0:48374:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.448993:0:48374:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x52:0x0](ffff9a78916f9148), inode mode 81a4 mask 40 00000080:00200008:1.0:1787827240.450723:0:48374:0:(file.c:2652:do_file_write_iter()) START file f2:[0x200001b72:0x52:0x0], ppos: 0, count: 1048576 00000080:00200000:1.0:1787827240.450734:0:48374:0:(file.c:2090:ll_file_io_generic()) f2: write ppos: 0, bytes: 1048576 00000080:00200000:1.0:1787827240.450739:0:48374:0:(file.c:1848:ll_io_set_mirror()) f2: desiginated mirror: 0 00000020:00200000:1.0:1787827240.450747:0:48374:0:(cl_io.c:218:cl_io_rw_init()) header@000000007c3c8b31[0x110, 1, [0x200001b72:0x52:0x0] hash] 00000020:00200000:1.0:1787827240.450749:0:48374:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 1048576) 0 0 00000080:00200000:1.0:1787827240.450753:0:48374:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x52:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.450769:0:48374:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 255] 00020000:00200000:1.0:1787827240.450777:0:48374:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 1048576) 1048576, 1048576 00020000:00200000:1.0:1787827240.450780:0:48374:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787827240.450794:0:48374:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 255] 00000008:00010000:1.0:1787827240.450809:0:48374:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:1.0:1787827240.450814:0:48374:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7886af0bb0, osc lock ffff9a789822a138, flags 0x0 00010000:00010000:1.0:1787827240.450883:0:48374:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.450895:0:48374:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.450908:0:48374:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827240.450928:0:48374:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ede00 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48374:1874671476530944:192.168.203.140@tcp:101:dd.0 00000100:00100000:1.0:1787827240.450985:0:48374:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.451998:0:48374:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ede00 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48374:1874671476530944:192.168.203.140@tcp:101:dd.0 00010000:00010000:1.0:1787827240.452009:0:48374:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.452025:0:48374:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.452034:0:48374:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.452044:0:48374:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.452054:0:48374:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,2 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.452067:0:48374:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.452073:0:48374:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.452090:0:48374:0:(vvp_io.c:1339:vvp_io_write_start()) f2: write [0, 1048576) append 0 append lockpos 0 00000080:00200000:1.0:1787827240.452103:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452142:0:48374:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452154:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d89501c3[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 0] 00000080:00200000:1.0:1787827240.452162:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008695d57a 17ffffc0001001 3:0 ffff9a7887aef828 0 lru 00000080:00200000:1.0:1787827240.452169:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a30eb0a5 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452171:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d89501c3 00000080:00200000:1.0:1787827240.452173:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:1.0:1787827240.452178:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452196:0:48374:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452200:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000580066a1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 1] 00000080:00200000:1.0:1787827240.452202:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000868aebfb 17ffffc0001001 3:0 ffff9a7887aef000 1 lru 00000080:00200000:1.0:1787827240.452206:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000071806c05 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452208:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000580066a1 00000080:00200000:1.0:1787827240.452209:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:1.0:1787827240.452210:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452220:0:48374:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452222:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f1cbc01d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 2] 00000080:00200000:1.0:1787827240.452224:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004ae9554b 17ffffc0001001 3:0 ffff9a7887aefcb0 2 lru 00000080:00200000:1.0:1787827240.452228:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000d56d2a2b 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452230:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f1cbc01d 00000080:00200000:1.0:1787827240.452231:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:1.0:1787827240.452232:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452241:0:48374:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452243:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000121a18ea[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 3] 00000080:00200000:1.0:1787827240.452246:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009386a846 17ffffc0001001 3:0 ffff9a7887aefae0 3 lru 00000080:00200000:1.0:1787827240.452249:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000de10ef49 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452251:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000121a18ea 00000080:00200000:1.0:1787827240.452252:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:1.0:1787827240.452253:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452261:0:48374:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452264:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ea9f4d30[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 4] 00000080:00200000:1.0:1787827240.452265:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ddea775e 17ffffc0001001 3:0 ffff9a7887aef658 4 lru 00000080:00200000:1.0:1787827240.452269:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000058c8759 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452270:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ea9f4d30 00000080:00200000:1.0:1787827240.452271:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:1.0:1787827240.452272:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452290:0:48374:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452293:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000001c37dcd[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 5] 00000080:00200000:1.0:1787827240.452295:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a63f6e50 17ffffc0001001 3:0 ffff9a7888756e80 5 lru 00000080:00200000:1.0:1787827240.452298:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004e482bf0 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452300:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000001c37dcd 00000080:00200000:1.0:1787827240.452301:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:1.0:1787827240.452302:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452310:0:48374:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452312:0:48374:0:(rw26.c:933:ll_write_end()) page@000000009ae89d41[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 6] 00000080:00200000:1.0:1787827240.452314:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e8dc0ffa 17ffffc0001001 3:0 ffff9a7888756488 6 lru 00000080:00200000:1.0:1787827240.452318:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000eb6f3271 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452319:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000009ae89d41 00000080:00200000:1.0:1787827240.452320:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:1.0:1787827240.452321:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452330:0:48374:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452332:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000082f01dc1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 7] 00000080:00200000:1.0:1787827240.452334:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006dd26639 17ffffc0001001 3:0 ffff9a7888756bc8 7 lru 00000080:00200000:1.0:1787827240.452339:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000df8cb23d 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452341:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000082f01dc1 00000080:00200000:1.0:1787827240.452342:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:1.0:1787827240.452343:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452352:0:48374:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452355:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008a4ab7de[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 8] 00000080:00200000:1.0:1787827240.452356:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006d3b21f2 17ffffc0001001 3:0 ffff9a7888756570 8 lru 00000080:00200000:1.0:1787827240.452360:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000016b6e6d4 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452362:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008a4ab7de 00000080:00200000:1.0:1787827240.452363:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:1.0:1787827240.452364:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452376:0:48374:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452378:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000015366a60[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 9] 00000080:00200000:1.0:1787827240.452380:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009b219a31 17ffffc0001021 2:0 ffff9a7888756828 9 lru 00000080:00200000:1.0:1787827240.452384:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003951c451 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452385:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000015366a60 00000080:00200000:1.0:1787827240.452386:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:1.0:1787827240.452387:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452396:0:48374:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452399:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000eb882da2[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 10] 00000080:00200000:1.0:1787827240.452401:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000058fb870e 17ffffc0001001 3:0 ffff9a7888756000 10 lru 00000080:00200000:1.0:1787827240.452405:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000007848fc58 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452406:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000eb882da2 00000080:00200000:1.0:1787827240.452407:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:1.0:1787827240.452408:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452416:0:48374:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452419:0:48374:0:(rw26.c:933:ll_write_end()) page@000000003a691dcc[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 11] 00000080:00200000:1.0:1787827240.452420:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b2f6f63c 17ffffc0001001 3:0 ffff9a7888756cb0 11 lru 00000080:00200000:1.0:1787827240.452424:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000015b3d0ce 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452425:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000003a691dcc 00000080:00200000:1.0:1787827240.452426:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:1.0:1787827240.452428:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452436:0:48374:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452439:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000deac0fa1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 12] 00000080:00200000:1.0:1787827240.452441:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a977b209 17ffffc0001001 3:0 ffff9a7888756ae0 12 lru 00000080:00200000:1.0:1787827240.452445:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000263508f3 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452447:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000deac0fa1 00000080:00200000:1.0:1787827240.452448:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:1.0:1787827240.452449:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452457:0:48374:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452460:0:48374:0:(rw26.c:933:ll_write_end()) page@000000009eca7b48[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 13] 00000080:00200000:1.0:1787827240.452461:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b18828fa 17ffffc0001001 3:0 ffff9a7888756658 13 lru 00000080:00200000:1.0:1787827240.452465:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000005f5f895a 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452467:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000009eca7b48 00000080:00200000:1.0:1787827240.452468:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:1.0:1787827240.452469:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452477:0:48374:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452479:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d76272e9[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 14] 00000080:00200000:1.0:1787827240.452481:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000003c6df473 17ffffc0001001 3:0 ffff9a78887569f8 14 lru 00000080:00200000:1.0:1787827240.452485:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000005986f257 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452486:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d76272e9 00000080:00200000:1.0:1787827240.452486:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:1.0:1787827240.452487:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452497:0:48374:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452499:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e58a6c29[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 15] 00000080:00200000:1.0:1787827240.452501:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000044df95de 17ffffc0001001 3:0 ffff9a7888756910 15 lru 00000080:00200000:1.0:1787827240.452504:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000566ea609 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452505:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e58a6c29 00000080:00200000:1.0:1787827240.452506:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:1.0:1787827240.452507:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452534:0:48374:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452537:0:48374:0:(rw26.c:933:ll_write_end()) page@000000007c0d0895[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 16] 00000080:00200000:1.0:1787827240.452539:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d637a3cd 17ffffc0001001 3:0 ffff9a7888756d98 16 lru 00000080:00200000:1.0:1787827240.452543:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004befce48 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452544:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000007c0d0895 00000080:00200000:1.0:1787827240.452545:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000080:00200000:1.0:1787827240.452546:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452556:0:48374:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452558:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000024ef9627[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 17] 00000080:00200000:1.0:1787827240.452560:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000000fce5963 17ffffc0001001 3:0 ffff9a78887563a0 17 lru 00000080:00200000:1.0:1787827240.452564:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000048282000 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452565:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000024ef9627 00000080:00200000:1.0:1787827240.452566:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:1.0:1787827240.452567:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452576:0:48374:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452578:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b6b40416[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 18] 00000080:00200000:1.0:1787827240.452580:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b69a8886 17ffffc0001001 3:0 ffff9a78887560e8 18 lru 00000080:00200000:1.0:1787827240.452584:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000331e0fc1 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452585:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b6b40416 00000080:00200000:1.0:1787827240.452586:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:1.0:1787827240.452588:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452596:0:48374:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452598:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000039a66bfd[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 19] 00000080:00200000:1.0:1787827240.452600:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c12eb74d 17ffffc0001001 3:0 ffff9a78887562b8 19 lru 00000080:00200000:1.0:1787827240.452604:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001b55b8b8 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452605:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000039a66bfd 00000080:00200000:1.0:1787827240.452606:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:1.0:1787827240.452607:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452615:0:48374:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452618:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002fabf6a1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 20] 00000080:00200000:1.0:1787827240.452620:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d4711176 17ffffc0001001 3:0 ffff9a78887561d0 20 lru 00000080:00200000:1.0:1787827240.452624:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000679f56ae 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452626:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002fabf6a1 00000080:00200000:1.0:1787827240.452626:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:1.0:1787827240.452628:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452636:0:48374:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452639:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000054bd5bb3[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 21] 00000080:00200000:1.0:1787827240.452641:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000009728599 17ffffc0001001 3:0 ffff9a7888756740 21 lru 00000080:00200000:1.0:1787827240.452645:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001586e2fd 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452646:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000054bd5bb3 00000080:00200000:1.0:1787827240.452647:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:1.0:1787827240.452648:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452665:0:48374:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452667:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ec186db5[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 22] 00000080:00200000:1.0:1787827240.452669:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d5c4fb93 17ffffc0001001 3:0 ffff9a7885a97658 22 lru 00000080:00200000:1.0:1787827240.452672:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a0fde55c 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452674:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ec186db5 00000080:00200000:1.0:1787827240.452675:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:1.0:1787827240.452677:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452686:0:48374:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452689:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008b388a80[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 23] 00000080:00200000:1.0:1787827240.452691:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000003499894a 17ffffc0001001 3:0 ffff9a7885a979f8 23 lru 00000080:00200000:1.0:1787827240.452695:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b91855b3 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452697:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008b388a80 00000080:00200000:1.0:1787827240.452697:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:1.0:1787827240.452698:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452709:0:48374:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452712:0:48374:0:(rw26.c:933:ll_write_end()) page@000000003fc89beb[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 24] 00000080:00200000:1.0:1787827240.452714:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b4ebd5c2 17ffffc0001021 2:0 ffff9a7885a97910 24 lru 00000080:00200000:1.0:1787827240.452717:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f03a475b 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452719:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000003fc89beb 00000080:00200000:1.0:1787827240.452720:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:1.0:1787827240.452721:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452730:0:48374:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452733:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006e0f4b5d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 25] 00000080:00200000:1.0:1787827240.452735:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000002b3c5be6 17ffffc0001001 3:0 ffff9a7885a97d98 25 lru 00000080:00200000:1.0:1787827240.452739:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003d10333c 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452741:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006e0f4b5d 00000080:00200000:1.0:1787827240.452742:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:1.0:1787827240.452743:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452752:0:48374:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452754:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fa10d3bf[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 26] 00000080:00200000:1.0:1787827240.452756:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000037840c72 17ffffc0001001 3:0 ffff9a7885a973a0 26 lru 00000080:00200000:1.0:1787827240.452760:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000031353d87 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452761:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fa10d3bf 00000080:00200000:1.0:1787827240.452762:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:1.0:1787827240.452763:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452772:0:48374:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452775:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b2ee9e7c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 27] 00000080:00200000:1.0:1787827240.452777:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e0ab0724 17ffffc0001001 3:0 ffff9a7885a970e8 27 lru 00000080:00200000:1.0:1787827240.452780:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c0f1701b 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452782:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b2ee9e7c 00000080:00200000:1.0:1787827240.452783:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:1.0:1787827240.452784:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452793:0:48374:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452795:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c4b5ff53[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 28] 00000080:00200000:1.0:1787827240.452797:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006e8288bc 17ffffc0001001 3:0 ffff9a7885a972b8 28 lru 00000080:00200000:1.0:1787827240.452801:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000009dce5da0 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452802:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c4b5ff53 00000080:00200000:1.0:1787827240.452803:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:1.0:1787827240.452804:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452814:0:48374:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452817:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c7cc853e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 29] 00000080:00200000:1.0:1787827240.452818:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000002c14f47a 17ffffc0001001 3:0 ffff9a7885a971d0 29 lru 00000080:00200000:1.0:1787827240.452823:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002e2f923f 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452826:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c7cc853e 00000080:00200000:1.0:1787827240.452850:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:1.0:1787827240.452854:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452873:0:48374:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452879:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e80e76e4[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 30] 00000080:00200000:1.0:1787827240.452882:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000058d6162a 17ffffc0001001 3:0 ffff9a7885a97740 30 lru 00000080:00200000:1.0:1787827240.452889:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004fc9f5c4 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452892:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e80e76e4 00000080:00200000:1.0:1787827240.452893:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:1.0:1787827240.452895:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452910:0:48374:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452914:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e390fb22[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 31] 00000080:00200000:1.0:1787827240.452916:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000fe988a38 17ffffc0001001 3:0 ffff9a7885a97e80 31 lru 00000080:00200000:1.0:1787827240.452920:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003aaa81f2 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452921:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e390fb22 00000080:00200000:1.0:1787827240.452922:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:1.0:1787827240.452924:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452933:0:48374:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452935:0:48374:0:(rw26.c:933:ll_write_end()) page@000000009895bfea[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 32] 00000080:00200000:1.0:1787827240.452937:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007f1dd57f 17ffffc0001001 3:0 ffff9a7885a97488 32 lru 00000080:00200000:1.0:1787827240.452941:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000afd68424 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452942:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000009895bfea 00000080:00200000:1.0:1787827240.452943:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:1.0:1787827240.452944:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452952:0:48374:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452955:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006938bc6b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 33] 00000080:00200000:1.0:1787827240.452957:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000081d578e3 17ffffc0001001 3:0 ffff9a7885a97bc8 33 lru 00000080:00200000:1.0:1787827240.452961:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000486f19e6 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452962:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006938bc6b 00000080:00200000:1.0:1787827240.452963:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:1.0:1787827240.452964:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452972:0:48374:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452975:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f2d4ab70[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 34] 00000080:00200000:1.0:1787827240.452977:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000738706f9 17ffffc0001001 3:0 ffff9a7885a97570 34 lru 00000080:00200000:1.0:1787827240.452981:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000400263b8 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.452983:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f2d4ab70 00000080:00200000:1.0:1787827240.452984:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:1.0:1787827240.452985:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.452993:0:48374:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:1.0:1787827240.452995:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c5ab2f65[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 35] 00000080:00200000:1.0:1787827240.452997:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005aab56ec 17ffffc0001001 3:0 ffff9a7885a97828 35 lru 00000080:00200000:1.0:1787827240.453000:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000a1aa6b5 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453002:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c5ab2f65 00000080:00200000:1.0:1787827240.453003:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:1.0:1787827240.453004:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453014:0:48374:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453016:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000026628db0[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 36] 00000080:00200000:1.0:1787827240.453018:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000076aa7404 17ffffc0001001 3:0 ffff9a7885a97000 36 lru 00000080:00200000:1.0:1787827240.453022:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000035bc8510 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453024:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000026628db0 00000080:00200000:1.0:1787827240.453025:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:1.0:1787827240.453026:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453034:0:48374:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453036:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ff07649d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 37] 00000080:00200000:1.0:1787827240.453038:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d82f0762 17ffffc0001001 3:0 ffff9a7885a97cb0 37 lru 00000080:00200000:1.0:1787827240.453042:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001eca7df2 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453043:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ff07649d 00000080:00200000:1.0:1787827240.453044:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:1.0:1787827240.453045:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453053:0:48374:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453056:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f419f01a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 38] 00000080:00200000:1.0:1787827240.453058:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000003a9cfb7c 17ffffc0001001 3:0 ffff9a7885a97ae0 38 lru 00000080:00200000:1.0:1787827240.453061:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003f8ed8d3 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453062:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f419f01a 00000080:00200000:1.0:1787827240.453063:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:1.0:1787827240.453065:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453082:0:48374:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453085:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000af0f2e32[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 39] 00000080:00200000:1.0:1787827240.453087:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000003f575158 17ffffc0001021 2:0 ffff9a7887843910 39 lru 00000080:00200000:1.0:1787827240.453090:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000009e0a6338 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453092:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000af0f2e32 00000080:00200000:1.0:1787827240.453093:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:1.0:1787827240.453094:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453103:0:48374:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453106:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000854a6b20[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 40] 00000080:00200000:1.0:1787827240.453108:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005711551d 17ffffc0001001 3:0 ffff9a7887843d98 40 lru 00000080:00200000:1.0:1787827240.453112:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000174c41dd 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453113:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000854a6b20 00000080:00200000:1.0:1787827240.453114:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:1.0:1787827240.453115:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453124:0:48374:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453126:0:48374:0:(rw26.c:933:ll_write_end()) page@000000001e1a6d74[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 41] 00000080:00200000:1.0:1787827240.453128:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000003395223 17ffffc0001001 3:0 ffff9a78878433a0 41 lru 00000080:00200000:1.0:1787827240.453131:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000ef975d3a 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453132:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000001e1a6d74 00000080:00200000:1.0:1787827240.453133:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:1.0:1787827240.453135:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453143:0:48374:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453146:0:48374:0:(rw26.c:933:ll_write_end()) page@000000007f86be09[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 42] 00000080:00200000:1.0:1787827240.453148:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b792a500 17ffffc0001001 3:0 ffff9a78878430e8 42 lru 00000080:00200000:1.0:1787827240.453152:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c9fcedbf 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453153:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000007f86be09 00000080:00200000:1.0:1787827240.453154:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:1.0:1787827240.453155:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453165:0:48374:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453168:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008d0706ab[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 43] 00000080:00200000:1.0:1787827240.453170:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001a3287e7 17ffffc0001001 3:0 ffff9a78878432b8 43 lru 00000080:00200000:1.0:1787827240.453174:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000062fcae70 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453175:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008d0706ab 00000080:00200000:1.0:1787827240.453176:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:1.0:1787827240.453177:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453185:0:48374:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453188:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c453f38f[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 44] 00000080:00200000:1.0:1787827240.453190:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000cfc662dd 17ffffc0001001 3:0 ffff9a78878431d0 44 lru 00000080:00200000:1.0:1787827240.453193:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003f93f6ea 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453194:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c453f38f 00000080:00200000:1.0:1787827240.453195:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:1.0:1787827240.453197:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453205:0:48374:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453208:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000097c469a3[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 45] 00000080:00200000:1.0:1787827240.453210:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c8be826b 17ffffc0001001 3:0 ffff9a7887843740 45 lru 00000080:00200000:1.0:1787827240.453214:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000fa917e84 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453215:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000097c469a3 00000080:00200000:1.0:1787827240.453216:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:1.0:1787827240.453217:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453225:0:48374:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453228:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002a52bb69[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 46] 00000080:00200000:1.0:1787827240.453229:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c8983f2d 17ffffc0001001 3:0 ffff9a7887843e80 46 lru 00000080:00200000:1.0:1787827240.453233:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000006b966215 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453234:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002a52bb69 00000080:00200000:1.0:1787827240.453235:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:1.0:1787827240.453236:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453245:0:48374:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453247:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000045b7069f[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 47] 00000080:00200000:1.0:1787827240.453249:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b9262620 17ffffc0001001 3:0 ffff9a7887843488 47 lru 00000080:00200000:1.0:1787827240.453253:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000099950986 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453255:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000045b7069f 00000080:00200000:1.0:1787827240.453256:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:1.0:1787827240.453257:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453265:0:48374:0:(rw26.c:875:ll_write_end()) pos 196608, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453268:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000639961ee[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 48] 00000080:00200000:1.0:1787827240.453270:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b965ca3c 17ffffc0001001 3:0 ffff9a7887843bc8 48 lru 00000080:00200000:1.0:1787827240.453274:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000042b9ef4d 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453275:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000639961ee 00000080:00200000:1.0:1787827240.453276:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:1.0:1787827240.453277:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 49 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453286:0:48374:0:(rw26.c:875:ll_write_end()) pos 200704, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453288:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000aaaa9143[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 49] 00000080:00200000:1.0:1787827240.453290:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000040f83ae2 17ffffc0001001 3:0 ffff9a7887843570 49 lru 00000080:00200000:1.0:1787827240.453293:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b49c2d6a 49: 1< 0 - - > 2< 200704 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453295:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000aaaa9143 00000080:00200000:1.0:1787827240.453295:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 50. 00000080:00200000:1.0:1787827240.453297:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 50 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453306:0:48374:0:(rw26.c:875:ll_write_end()) pos 204800, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453309:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000afaffdcb[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 50] 00000080:00200000:1.0:1787827240.453311:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a9b5c228 17ffffc0001001 3:0 ffff9a7887843828 50 lru 00000080:00200000:1.0:1787827240.453315:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000043ef977f 50: 1< 0 - - > 2< 204800 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453317:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000afaffdcb 00000080:00200000:1.0:1787827240.453317:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 51. 00000080:00200000:1.0:1787827240.453319:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 51 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453327:0:48374:0:(rw26.c:875:ll_write_end()) pos 208896, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453329:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000bf07198e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 51] 00000080:00200000:1.0:1787827240.453331:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001f6dc8de 17ffffc0001001 3:0 ffff9a7887843000 51 lru 00000080:00200000:1.0:1787827240.453334:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000000854155 51: 1< 0 - - > 2< 208896 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453336:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000bf07198e 00000080:00200000:1.0:1787827240.453337:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 52. 00000080:00200000:1.0:1787827240.453338:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 52 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453346:0:48374:0:(rw26.c:875:ll_write_end()) pos 212992, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453349:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004bf02787[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 52] 00000080:00200000:1.0:1787827240.453351:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000f00c9ac9 17ffffc0001001 3:0 ffff9a7887843cb0 52 lru 00000080:00200000:1.0:1787827240.453355:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000fccca445 52: 1< 0 - - > 2< 212992 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453357:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004bf02787 00000080:00200000:1.0:1787827240.453358:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 53. 00000080:00200000:1.0:1787827240.453359:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 53 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453367:0:48374:0:(rw26.c:875:ll_write_end()) pos 217088, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453370:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000273a1611[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 53] 00000080:00200000:1.0:1787827240.453372:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001ddacbd0 17ffffc0001001 3:0 ffff9a7887843ae0 53 lru 00000080:00200000:1.0:1787827240.453375:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000034622e82 53: 1< 0 - - > 2< 217088 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453377:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000273a1611 00000080:00200000:1.0:1787827240.453377:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 54. 00000080:00200000:1.0:1787827240.453379:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 54 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453389:0:48374:0:(rw26.c:875:ll_write_end()) pos 221184, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453392:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002380c3a7[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 54] 00000080:00200000:1.0:1787827240.453394:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000969eb8ba 17ffffc0001021 2:0 ffff9a7887843658 54 lru 00000080:00200000:1.0:1787827240.453397:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000006beb328d 54: 1< 0 - - > 2< 221184 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453398:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002380c3a7 00000080:00200000:1.0:1787827240.453399:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 55. 00000080:00200000:1.0:1787827240.453401:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 55 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453409:0:48374:0:(rw26.c:875:ll_write_end()) pos 225280, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453411:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c9719f8a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 55] 00000080:00200000:1.0:1787827240.453413:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000006663d8f 17ffffc0001001 3:0 ffff9a78878439f8 55 lru 00000080:00200000:1.0:1787827240.453417:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000071e9b493 55: 1< 0 - - > 2< 225280 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453419:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c9719f8a 00000080:00200000:1.0:1787827240.453420:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 56. 00000080:00200000:1.0:1787827240.453421:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 56 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453438:0:48374:0:(rw26.c:875:ll_write_end()) pos 229376, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453441:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000772e80c9[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 56] 00000080:00200000:1.0:1787827240.453443:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000063ff6098 17ffffc0001001 3:0 ffff9a7886b7dcb0 56 lru 00000080:00200000:1.0:1787827240.453446:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f111d841 56: 1< 0 - - > 2< 229376 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453448:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000772e80c9 00000080:00200000:1.0:1787827240.453449:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 57. 00000080:00200000:1.0:1787827240.453450:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 57 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453460:0:48374:0:(rw26.c:875:ll_write_end()) pos 233472, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453463:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fcc0942f[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 57] 00000080:00200000:1.0:1787827240.453465:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000679d012e 17ffffc0001001 3:0 ffff9a7886b7dae0 57 lru 00000080:00200000:1.0:1787827240.453468:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000687b0298 57: 1< 0 - - > 2< 233472 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453470:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fcc0942f 00000080:00200000:1.0:1787827240.453471:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 58. 00000080:00200000:1.0:1787827240.453472:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 58 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453481:0:48374:0:(rw26.c:875:ll_write_end()) pos 237568, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453484:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000adaca424[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 58] 00000080:00200000:1.0:1787827240.453486:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000030b4c713 17ffffc0001001 3:0 ffff9a7886b7d658 58 lru 00000080:00200000:1.0:1787827240.453489:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000bf2d23cd 58: 1< 0 - - > 2< 237568 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453491:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000adaca424 00000080:00200000:1.0:1787827240.453492:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 59. 00000080:00200000:1.0:1787827240.453493:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 59 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453501:0:48374:0:(rw26.c:875:ll_write_end()) pos 241664, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453504:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e3b095f7[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 59] 00000080:00200000:1.0:1787827240.453506:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000048ae1c0f 17ffffc0001001 3:0 ffff9a7886b7d9f8 59 lru 00000080:00200000:1.0:1787827240.453525:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b365fdb9 59: 1< 0 - - > 2< 241664 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453527:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e3b095f7 00000080:00200000:1.0:1787827240.453528:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 60. 00000080:00200000:1.0:1787827240.453529:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 60 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453541:0:48374:0:(rw26.c:875:ll_write_end()) pos 245760, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453544:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fce29191[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 60] 00000080:00200000:1.0:1787827240.453546:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000067882074 17ffffc0001001 3:0 ffff9a7886b7d910 60 lru 00000080:00200000:1.0:1787827240.453550:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000326c077 60: 1< 0 - - > 2< 245760 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453552:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fce29191 00000080:00200000:1.0:1787827240.453553:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 61. 00000080:00200000:1.0:1787827240.453554:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 61 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453562:0:48374:0:(rw26.c:875:ll_write_end()) pos 249856, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453565:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000054eb82be[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 61] 00000080:00200000:1.0:1787827240.453567:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a54147f0 17ffffc0001001 3:0 ffff9a7886b7dd98 61 lru 00000080:00200000:1.0:1787827240.453571:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000007a2d1027 61: 1< 0 - - > 2< 249856 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453573:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000054eb82be 00000080:00200000:1.0:1787827240.453573:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 62. 00000080:00200000:1.0:1787827240.453575:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 62 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453582:0:48374:0:(rw26.c:875:ll_write_end()) pos 253952, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453585:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000cf5cd0c1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 62] 00000080:00200000:1.0:1787827240.453587:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000fd173e9b 17ffffc0001001 3:0 ffff9a7886b7d3a0 62 lru 00000080:00200000:1.0:1787827240.453590:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f21081ad 62: 1< 0 - - > 2< 253952 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453592:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000cf5cd0c1 00000080:00200000:1.0:1787827240.453593:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 63. 00000080:00200000:1.0:1787827240.453594:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 63 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453603:0:48374:0:(rw26.c:875:ll_write_end()) pos 258048, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453606:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e17f301c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 63] 00000080:00200000:1.0:1787827240.453608:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000094f814dc 17ffffc0001001 3:0 ffff9a7886b7d0e8 63 lru 00000080:00200000:1.0:1787827240.453611:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000008887fc68 63: 1< 0 - - > 2< 258048 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453613:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e17f301c 00000080:00200000:1.0:1787827240.453614:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 64. 00000080:00200000:1.0:1787827240.453615:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 64 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453627:0:48374:0:(rw26.c:875:ll_write_end()) pos 262144, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453630:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000409903fc[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 64] 00000080:00200000:1.0:1787827240.453632:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000982e7257 17ffffc0001001 3:0 ffff9a7886b7d2b8 64 lru 00000080:00200000:1.0:1787827240.453635:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000413d2752 64: 1< 0 - - > 2< 262144 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453637:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000409903fc 00000080:00200000:1.0:1787827240.453638:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 65. 00000080:00200000:1.0:1787827240.453639:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 65 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453648:0:48374:0:(rw26.c:875:ll_write_end()) pos 266240, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453651:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000004d0afff[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 65] 00000080:00200000:1.0:1787827240.453653:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004fd6453b 17ffffc0001001 3:0 ffff9a7886b7d1d0 65 lru 00000080:00200000:1.0:1787827240.453656:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b304f2ba 65: 1< 0 - - > 2< 266240 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453658:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000004d0afff 00000080:00200000:1.0:1787827240.453659:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 66. 00000080:00200000:1.0:1787827240.453661:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 66 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453669:0:48374:0:(rw26.c:875:ll_write_end()) pos 270336, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453672:0:48374:0:(rw26.c:933:ll_write_end()) page@000000009fdb4202[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 66] 00000080:00200000:1.0:1787827240.453674:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a7e0ce73 17ffffc0001001 3:0 ffff9a7886b7d740 66 lru 00000080:00200000:1.0:1787827240.453678:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b5c6d8e1 66: 1< 0 - - > 2< 270336 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453680:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000009fdb4202 00000080:00200000:1.0:1787827240.453680:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 67. 00000080:00200000:1.0:1787827240.453681:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 67 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453690:0:48374:0:(rw26.c:875:ll_write_end()) pos 274432, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453692:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ec52fdec[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 67] 00000080:00200000:1.0:1787827240.453694:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a4767aac 17ffffc0001001 3:0 ffff9a7886b7de80 67 lru 00000080:00200000:1.0:1787827240.453698:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000cc393ffc 67: 1< 0 - - > 2< 274432 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453699:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ec52fdec 00000080:00200000:1.0:1787827240.453700:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 68. 00000080:00200000:1.0:1787827240.453701:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 68 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453709:0:48374:0:(rw26.c:875:ll_write_end()) pos 278528, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453711:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000a162f5e8[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 68] 00000080:00200000:1.0:1787827240.453713:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c3c0aa2d 17ffffc0001001 3:0 ffff9a7886b7d488 68 lru 00000080:00200000:1.0:1787827240.453717:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000036482c07 68: 1< 0 - - > 2< 278528 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453718:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000a162f5e8 00000080:00200000:1.0:1787827240.453719:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 69. 00000080:00200000:1.0:1787827240.453720:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 69 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453737:0:48374:0:(rw26.c:875:ll_write_end()) pos 282624, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453740:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b8edad70[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 69] 00000080:00200000:1.0:1787827240.453745:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000014ccd246 17ffffc0001021 2:0 ffff9a7886b7dbc8 69 lru 00000080:00200000:1.0:1787827240.453749:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000d2a69aa5 69: 1< 0 - - > 2< 282624 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453751:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b8edad70 00000080:00200000:1.0:1787827240.453752:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 70. 00000080:00200000:1.0:1787827240.453754:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 70 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453768:0:48374:0:(rw26.c:875:ll_write_end()) pos 286720, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453774:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000cc114daf[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 70] 00000080:00200000:1.0:1787827240.453777:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000002930e421 17ffffc0001001 3:0 ffff9a7886b7d570 70 lru 00000080:00200000:1.0:1787827240.453784:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000fa6bbc1e 70: 1< 0 - - > 2< 286720 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453786:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000cc114daf 00000080:00200000:1.0:1787827240.453787:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 71. 00000080:00200000:1.0:1787827240.453789:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 71 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453806:0:48374:0:(rw26.c:875:ll_write_end()) pos 290816, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453811:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000038ea2dd7[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 71] 00000080:00200000:1.0:1787827240.453813:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000000761dae9 17ffffc0001001 3:0 ffff9a7886b7d828 71 lru 00000080:00200000:1.0:1787827240.453819:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001faf7f3b 71: 1< 0 - - > 2< 290816 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453821:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000038ea2dd7 00000080:00200000:1.0:1787827240.453823:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 72. 00000080:00200000:1.0:1787827240.453825:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 72 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453862:0:48374:0:(rw26.c:875:ll_write_end()) pos 294912, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453868:0:48374:0:(rw26.c:933:ll_write_end()) page@000000001cc7cd6c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 72] 00000080:00200000:1.0:1787827240.453871:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000bb4a5016 17ffffc0001001 3:0 ffff9a7886b7d000 72 lru 00000080:00200000:1.0:1787827240.453876:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c8b42669 72: 1< 0 - - > 2< 294912 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453878:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000001cc7cd6c 00000080:00200000:1.0:1787827240.453880:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 73. 00000080:00200000:1.0:1787827240.453882:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 73 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453903:0:48374:0:(rw26.c:875:ll_write_end()) pos 299008, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453908:0:48374:0:(rw26.c:933:ll_write_end()) page@000000009fcc2af8[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 73] 00000080:00200000:1.0:1787827240.453911:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e0572506 17ffffc0001001 3:0 ffff9a7888caa828 73 lru 00000080:00200000:1.0:1787827240.453916:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f13d578f 73: 1< 0 - - > 2< 299008 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453918:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000009fcc2af8 00000080:00200000:1.0:1787827240.453920:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 74. 00000080:00200000:1.0:1787827240.453921:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 74 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453936:0:48374:0:(rw26.c:875:ll_write_end()) pos 303104, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453941:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b9ab8745[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 74] 00000080:00200000:1.0:1787827240.453944:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005c32ce76 17ffffc0001001 3:0 ffff9a7888caa000 74 lru 00000080:00200000:1.0:1787827240.453951:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000051036abd 74: 1< 0 - - > 2< 303104 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453953:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b9ab8745 00000080:00200000:1.0:1787827240.453955:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 75. 00000080:00200000:1.0:1787827240.453957:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 75 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453972:0:48374:0:(rw26.c:875:ll_write_end()) pos 307200, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453975:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006809032b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 75] 00000080:00200000:1.0:1787827240.453977:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000019d1a11c 17ffffc0001001 3:0 ffff9a7888caacb0 75 lru 00000080:00200000:1.0:1787827240.453982:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003a329913 75: 1< 0 - - > 2< 307200 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.453983:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006809032b 00000080:00200000:1.0:1787827240.453984:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 76. 00000080:00200000:1.0:1787827240.453986:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 76 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.453995:0:48374:0:(rw26.c:875:ll_write_end()) pos 311296, len 4096, copied 4096 00000080:00200000:1.0:1787827240.453998:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000cbfaffda[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 76] 00000080:00200000:1.0:1787827240.454000:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001f2b0fe9 17ffffc0001001 3:0 ffff9a7888caaae0 76 lru 00000080:00200000:1.0:1787827240.454004:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000097a545a6 76: 1< 0 - - > 2< 311296 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454005:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000cbfaffda 00000080:00200000:1.0:1787827240.454006:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 77. 00000080:00200000:1.0:1787827240.454007:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 77 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454015:0:48374:0:(rw26.c:875:ll_write_end()) pos 315392, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454018:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d6f99c6f[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 77] 00000080:00200000:1.0:1787827240.454020:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000027a1003a 17ffffc0001001 3:0 ffff9a7888caa658 77 lru 00000080:00200000:1.0:1787827240.454023:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c8f500b2 77: 1< 0 - - > 2< 315392 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454024:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d6f99c6f 00000080:00200000:1.0:1787827240.454025:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 78. 00000080:00200000:1.0:1787827240.454026:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 78 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454036:0:48374:0:(rw26.c:875:ll_write_end()) pos 319488, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454039:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000001b42f10[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 78] 00000080:00200000:1.0:1787827240.454041:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001fad60b3 17ffffc0001001 3:0 ffff9a7888caa9f8 78 lru 00000080:00200000:1.0:1787827240.454045:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002d5177f2 78: 1< 0 - - > 2< 319488 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454046:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000001b42f10 00000080:00200000:1.0:1787827240.454047:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 79. 00000080:00200000:1.0:1787827240.454048:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 79 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454056:0:48374:0:(rw26.c:875:ll_write_end()) pos 323584, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454059:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000a55cf9a5[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 79] 00000080:00200000:1.0:1787827240.454061:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b63f0ce2 17ffffc0001001 3:0 ffff9a7888caa910 79 lru 00000080:00200000:1.0:1787827240.454065:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002a7d2dd4 79: 1< 0 - - > 2< 323584 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454067:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000a55cf9a5 00000080:00200000:1.0:1787827240.454068:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 80. 00000080:00200000:1.0:1787827240.454069:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 80 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454077:0:48374:0:(rw26.c:875:ll_write_end()) pos 327680, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454080:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004010e952[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 80] 00000080:00200000:1.0:1787827240.454081:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000045b537a7 17ffffc0001001 3:0 ffff9a7888caad98 80 lru 00000080:00200000:1.0:1787827240.454085:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000105bff9b 80: 1< 0 - - > 2< 327680 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454086:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004010e952 00000080:00200000:1.0:1787827240.454087:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 81. 00000080:00200000:1.0:1787827240.454088:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 81 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454096:0:48374:0:(rw26.c:875:ll_write_end()) pos 331776, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454099:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000114bd13c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 81] 00000080:00200000:1.0:1787827240.454101:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000000899a418 17ffffc0001001 3:0 ffff9a7888caa3a0 81 lru 00000080:00200000:1.0:1787827240.454104:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000007e1eb017 81: 1< 0 - - > 2< 331776 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454105:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000114bd13c 00000080:00200000:1.0:1787827240.454106:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 82. 00000080:00200000:1.0:1787827240.454107:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 82 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454115:0:48374:0:(rw26.c:875:ll_write_end()) pos 335872, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454118:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000000c6c10c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 82] 00000080:00200000:1.0:1787827240.454120:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b0ba6e2e 17ffffc0001001 3:0 ffff9a7888caa0e8 82 lru 00000080:00200000:1.0:1787827240.454123:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000000234190 82: 1< 0 - - > 2< 335872 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454124:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000000c6c10c 00000080:00200000:1.0:1787827240.454125:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 83. 00000080:00200000:1.0:1787827240.454126:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 83 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454134:0:48374:0:(rw26.c:875:ll_write_end()) pos 339968, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454137:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c543349e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 83] 00000080:00200000:1.0:1787827240.454139:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000076bbc2b8 17ffffc0001001 3:0 ffff9a7888caa2b8 83 lru 00000080:00200000:1.0:1787827240.454143:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002082dd53 83: 1< 0 - - > 2< 339968 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454144:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c543349e 00000080:00200000:1.0:1787827240.454145:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 84. 00000080:00200000:1.0:1787827240.454146:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 84 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454157:0:48374:0:(rw26.c:875:ll_write_end()) pos 344064, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454160:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e510f808[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 84] 00000080:00200000:1.0:1787827240.454162:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000095ff537 17ffffc0001021 2:0 ffff9a7888caa1d0 84 lru 00000080:00200000:1.0:1787827240.454166:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000086978446 84: 1< 0 - - > 2< 344064 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454168:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e510f808 00000080:00200000:1.0:1787827240.454169:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 85. 00000080:00200000:1.0:1787827240.454170:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 85 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454179:0:48374:0:(rw26.c:875:ll_write_end()) pos 348160, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454182:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000eed139e7[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 85] 00000080:00200000:1.0:1787827240.454184:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c148724f 17ffffc0001001 3:0 ffff9a7888caa740 85 lru 00000080:00200000:1.0:1787827240.454187:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000627a5a7f 85: 1< 0 - - > 2< 348160 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454188:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000eed139e7 00000080:00200000:1.0:1787827240.454189:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 86. 00000080:00200000:1.0:1787827240.454190:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 86 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454198:0:48374:0:(rw26.c:875:ll_write_end()) pos 352256, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454201:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d00fec3d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 86] 00000080:00200000:1.0:1787827240.454203:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000016d849dd 17ffffc0001001 3:0 ffff9a7888caae80 86 lru 00000080:00200000:1.0:1787827240.454206:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000134056bd 86: 1< 0 - - > 2< 352256 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454208:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d00fec3d 00000080:00200000:1.0:1787827240.454209:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 87. 00000080:00200000:1.0:1787827240.454210:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 87 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454218:0:48374:0:(rw26.c:875:ll_write_end()) pos 356352, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454221:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000022497752[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 87] 00000080:00200000:1.0:1787827240.454223:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000bd9ef33e 17ffffc0001001 3:0 ffff9a7888caa488 87 lru 00000080:00200000:1.0:1787827240.454226:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000005dde4977 87: 1< 0 - - > 2< 356352 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454228:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000022497752 00000080:00200000:1.0:1787827240.454229:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 88. 00000080:00200000:1.0:1787827240.454230:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 88 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454237:0:48374:0:(rw26.c:875:ll_write_end()) pos 360448, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454240:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000060a93607[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 88] 00000080:00200000:1.0:1787827240.454242:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007102b71d 17ffffc0001001 3:0 ffff9a7888caabc8 88 lru 00000080:00200000:1.0:1787827240.454246:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003aeb6b40 88: 1< 0 - - > 2< 360448 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454248:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000060a93607 00000080:00200000:1.0:1787827240.454249:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 89. 00000080:00200000:1.0:1787827240.454250:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 89 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454258:0:48374:0:(rw26.c:875:ll_write_end()) pos 364544, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454260:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f8c80c78[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 89] 00000080:00200000:1.0:1787827240.454262:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000033247e1f 17ffffc0001001 3:0 ffff9a7888caa570 89 lru 00000080:00200000:1.0:1787827240.454265:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001783fcb7 89: 1< 0 - - > 2< 364544 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454267:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f8c80c78 00000080:00200000:1.0:1787827240.454268:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 90. 00000080:00200000:1.0:1787827240.454269:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 90 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454284:0:48374:0:(rw26.c:875:ll_write_end()) pos 368640, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454287:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000005176e6c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 90] 00000080:00200000:1.0:1787827240.454289:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000003a68ae47 17ffffc0001001 3:0 ffff9a788780a658 90 lru 00000080:00200000:1.0:1787827240.454293:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003175888d 90: 1< 0 - - > 2< 368640 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454294:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000005176e6c 00000080:00200000:1.0:1787827240.454295:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 91. 00000080:00200000:1.0:1787827240.454296:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 91 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454305:0:48374:0:(rw26.c:875:ll_write_end()) pos 372736, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454307:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d62a6d84[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 91] 00000080:00200000:1.0:1787827240.454309:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b9859206 17ffffc0001001 3:0 ffff9a788780a9f8 91 lru 00000080:00200000:1.0:1787827240.454312:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000a44a2dd 91: 1< 0 - - > 2< 372736 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454314:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d62a6d84 00000080:00200000:1.0:1787827240.454315:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 92. 00000080:00200000:1.0:1787827240.454316:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 92 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454325:0:48374:0:(rw26.c:875:ll_write_end()) pos 376832, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454328:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000059134e10[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 92] 00000080:00200000:1.0:1787827240.454329:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008e58d279 17ffffc0001001 3:0 ffff9a788780a910 92 lru 00000080:00200000:1.0:1787827240.454333:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c66eedaa 92: 1< 0 - - > 2< 376832 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454335:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000059134e10 00000080:00200000:1.0:1787827240.454336:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 93. 00000080:00200000:1.0:1787827240.454337:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 93 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454346:0:48374:0:(rw26.c:875:ll_write_end()) pos 380928, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454349:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f5d35449[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 93] 00000080:00200000:1.0:1787827240.454351:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000780c8c42 17ffffc0001001 3:0 ffff9a788780ad98 93 lru 00000080:00200000:1.0:1787827240.454355:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004be6cc93 93: 1< 0 - - > 2< 380928 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454356:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f5d35449 00000080:00200000:1.0:1787827240.454357:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 94. 00000080:00200000:1.0:1787827240.454358:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 94 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454366:0:48374:0:(rw26.c:875:ll_write_end()) pos 385024, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454369:0:48374:0:(rw26.c:933:ll_write_end()) page@000000007639e35a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 94] 00000080:00200000:1.0:1787827240.454371:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a54c553a 17ffffc0001001 3:0 ffff9a788780a3a0 94 lru 00000080:00200000:1.0:1787827240.454374:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000034470407 94: 1< 0 - - > 2< 385024 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454375:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000007639e35a 00000080:00200000:1.0:1787827240.454376:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 95. 00000080:00200000:1.0:1787827240.454378:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 95 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454386:0:48374:0:(rw26.c:875:ll_write_end()) pos 389120, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454389:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ce209b7a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 95] 00000080:00200000:1.0:1787827240.454391:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009b8c4f49 17ffffc0001001 3:0 ffff9a788780a0e8 95 lru 00000080:00200000:1.0:1787827240.454395:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a4fb0d28 95: 1< 0 - - > 2< 389120 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454397:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ce209b7a 00000080:00200000:1.0:1787827240.454398:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 96. 00000080:00200000:1.0:1787827240.454399:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 96 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454407:0:48374:0:(rw26.c:875:ll_write_end()) pos 393216, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454409:0:48374:0:(rw26.c:933:ll_write_end()) page@000000007fa78529[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 96] 00000080:00200000:1.0:1787827240.454411:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005b36a6ca 17ffffc0001001 3:0 ffff9a788780a2b8 96 lru 00000080:00200000:1.0:1787827240.454414:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000dfb932f9 96: 1< 0 - - > 2< 393216 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454415:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000007fa78529 00000080:00200000:1.0:1787827240.454416:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 97. 00000080:00200000:1.0:1787827240.454417:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 97 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454425:0:48374:0:(rw26.c:875:ll_write_end()) pos 397312, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454428:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000353916ea[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 97] 00000080:00200000:1.0:1787827240.454430:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007046be99 17ffffc0001001 3:0 ffff9a788780a1d0 97 lru 00000080:00200000:1.0:1787827240.454434:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000038481017 97: 1< 0 - - > 2< 397312 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454435:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000353916ea 00000080:00200000:1.0:1787827240.454436:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 98. 00000080:00200000:1.0:1787827240.454437:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 98 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454445:0:48374:0:(rw26.c:875:ll_write_end()) pos 401408, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454448:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000605db852[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 98] 00000080:00200000:1.0:1787827240.454449:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000036f857bf 17ffffc0001001 3:0 ffff9a788780a740 98 lru 00000080:00200000:1.0:1787827240.454453:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a215215b 98: 1< 0 - - > 2< 401408 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454454:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000605db852 00000080:00200000:1.0:1787827240.454455:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 99. 00000080:00200000:1.0:1787827240.454457:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 99 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454469:0:48374:0:(rw26.c:875:ll_write_end()) pos 405504, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454471:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000092f5a96c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 99] 00000080:00200000:1.0:1787827240.454474:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000943af103 17ffffc0001021 2:0 ffff9a788780ae80 99 lru 00000080:00200000:1.0:1787827240.454477:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000093b3c8da 99: 1< 0 - - > 2< 405504 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454479:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000092f5a96c 00000080:00200000:1.0:1787827240.454480:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 100. 00000080:00200000:1.0:1787827240.454481:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 100 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454489:0:48374:0:(rw26.c:875:ll_write_end()) pos 409600, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454491:0:48374:0:(rw26.c:933:ll_write_end()) page@000000000330a2ca[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 100] 00000080:00200000:1.0:1787827240.454493:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000f85a75e3 17ffffc0001001 3:0 ffff9a788780a488 100 lru 00000080:00200000:1.0:1787827240.454497:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000169c11c1 100: 1< 0 - - > 2< 409600 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454498:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000000330a2ca 00000080:00200000:1.0:1787827240.454499:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 101. 00000080:00200000:1.0:1787827240.454500:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 101 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454521:0:48374:0:(rw26.c:875:ll_write_end()) pos 413696, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454525:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ff2c6ea0[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 101] 00000080:00200000:1.0:1787827240.454527:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000278a6826 17ffffc0001001 3:0 ffff9a788780abc8 101 lru 00000080:00200000:1.0:1787827240.454531:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000ff3dd372 101: 1< 0 - - > 2< 413696 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454532:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ff2c6ea0 00000080:00200000:1.0:1787827240.454533:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 102. 00000080:00200000:1.0:1787827240.454535:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 102 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454545:0:48374:0:(rw26.c:875:ll_write_end()) pos 417792, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454548:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000075cef713[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 102] 00000080:00200000:1.0:1787827240.454550:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000037133aeb 17ffffc0001001 3:0 ffff9a788780a570 102 lru 00000080:00200000:1.0:1787827240.454554:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000d5dee9f7 102: 1< 0 - - > 2< 417792 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454556:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000075cef713 00000080:00200000:1.0:1787827240.454557:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 103. 00000080:00200000:1.0:1787827240.454557:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 103 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454566:0:48374:0:(rw26.c:875:ll_write_end()) pos 421888, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454568:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000067614c11[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 103] 00000080:00200000:1.0:1787827240.454570:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007d49aedd 17ffffc0001001 3:0 ffff9a788780a828 103 lru 00000080:00200000:1.0:1787827240.454574:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a798b5de 103: 1< 0 - - > 2< 421888 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454575:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000067614c11 00000080:00200000:1.0:1787827240.454576:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 104. 00000080:00200000:1.0:1787827240.454577:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 104 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454585:0:48374:0:(rw26.c:875:ll_write_end()) pos 425984, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454588:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008db18f8d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 104] 00000080:00200000:1.0:1787827240.454590:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000092c52c6e 17ffffc0001001 3:0 ffff9a788780a000 104 lru 00000080:00200000:1.0:1787827240.454594:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b9937472 104: 1< 0 - - > 2< 425984 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454596:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008db18f8d 00000080:00200000:1.0:1787827240.454596:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 105. 00000080:00200000:1.0:1787827240.454597:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 105 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454605:0:48374:0:(rw26.c:875:ll_write_end()) pos 430080, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454608:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d86fb5b5[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 105] 00000080:00200000:1.0:1787827240.454610:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007ab72c9e 17ffffc0001001 3:0 ffff9a788780acb0 105 lru 00000080:00200000:1.0:1787827240.454613:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000996f309c 105: 1< 0 - - > 2< 430080 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454615:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d86fb5b5 00000080:00200000:1.0:1787827240.454615:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 106. 00000080:00200000:1.0:1787827240.454617:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 106 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454626:0:48374:0:(rw26.c:875:ll_write_end()) pos 434176, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454628:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000077da3b49[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 106] 00000080:00200000:1.0:1787827240.454631:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000947e661f 17ffffc0001001 3:0 ffff9a788780aae0 106 lru 00000080:00200000:1.0:1787827240.454634:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000e25ad53 106: 1< 0 - - > 2< 434176 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454636:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000077da3b49 00000080:00200000:1.0:1787827240.454637:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 107. 00000080:00200000:1.0:1787827240.454638:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 107 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454653:0:48374:0:(rw26.c:875:ll_write_end()) pos 438272, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454656:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000df2b600e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 107] 00000080:00200000:1.0:1787827240.454658:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000005b0d214 17ffffc0001001 3:0 ffff9a78870c92b8 107 lru 00000080:00200000:1.0:1787827240.454661:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000008631d98d 107: 1< 0 - - > 2< 438272 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454663:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000df2b600e 00000080:00200000:1.0:1787827240.454664:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 108. 00000080:00200000:1.0:1787827240.454665:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 108 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454673:0:48374:0:(rw26.c:875:ll_write_end()) pos 442368, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454676:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000da3d678a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 108] 00000080:00200000:1.0:1787827240.454678:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ab78492b 17ffffc0001001 3:0 ffff9a78870c91d0 108 lru 00000080:00200000:1.0:1787827240.454682:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000007ad6c16b 108: 1< 0 - - > 2< 442368 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454684:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000da3d678a 00000080:00200000:1.0:1787827240.454685:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 109. 00000080:00200000:1.0:1787827240.454686:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 109 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454695:0:48374:0:(rw26.c:875:ll_write_end()) pos 446464, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454697:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000dc14355c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 109] 00000080:00200000:1.0:1787827240.454699:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c4e4bf12 17ffffc0001001 3:0 ffff9a78870c9740 109 lru 00000080:00200000:1.0:1787827240.454702:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000091bc1a84 109: 1< 0 - - > 2< 446464 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454704:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000dc14355c 00000080:00200000:1.0:1787827240.454705:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 110. 00000080:00200000:1.0:1787827240.454706:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 110 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454714:0:48374:0:(rw26.c:875:ll_write_end()) pos 450560, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454717:0:48374:0:(rw26.c:933:ll_write_end()) page@000000003cda13d7[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 110] 00000080:00200000:1.0:1787827240.454719:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000073cb3f25 17ffffc0001001 3:0 ffff9a78870c9e80 110 lru 00000080:00200000:1.0:1787827240.454723:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000008108fe2b 110: 1< 0 - - > 2< 450560 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454725:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000003cda13d7 00000080:00200000:1.0:1787827240.454726:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 111. 00000080:00200000:1.0:1787827240.454727:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 111 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454735:0:48374:0:(rw26.c:875:ll_write_end()) pos 454656, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454738:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000900209a0[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 111] 00000080:00200000:1.0:1787827240.454740:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001e50e3a3 17ffffc0001001 3:0 ffff9a78870c9488 111 lru 00000080:00200000:1.0:1787827240.454743:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001a21349c 111: 1< 0 - - > 2< 454656 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454744:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000900209a0 00000080:00200000:1.0:1787827240.454745:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 112. 00000080:00200000:1.0:1787827240.454746:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 112 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454755:0:48374:0:(rw26.c:875:ll_write_end()) pos 458752, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454757:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006e7f73b2[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 112] 00000080:00200000:1.0:1787827240.454759:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009d306280 17ffffc0001001 3:0 ffff9a78870c9bc8 112 lru 00000080:00200000:1.0:1787827240.454762:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000a22c871 112: 1< 0 - - > 2< 458752 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454763:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006e7f73b2 00000080:00200000:1.0:1787827240.454764:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 113. 00000080:00200000:1.0:1787827240.454766:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 113 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454775:0:48374:0:(rw26.c:875:ll_write_end()) pos 462848, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454778:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000293e2aca[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 113] 00000080:00200000:1.0:1787827240.454780:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000062354062 17ffffc0001001 3:0 ffff9a78870c9570 113 lru 00000080:00200000:1.0:1787827240.454784:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000008e16e333 113: 1< 0 - - > 2< 462848 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454785:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000293e2aca 00000080:00200000:1.0:1787827240.454786:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 114. 00000080:00200000:1.0:1787827240.454787:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 114 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454798:0:48374:0:(rw26.c:875:ll_write_end()) pos 466944, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454801:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b049ba53[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 114] 00000080:00200000:1.0:1787827240.454803:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008f19a688 17ffffc0001021 2:0 ffff9a78870c9828 114 lru 00000080:00200000:1.0:1787827240.454806:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a8055a03 114: 1< 0 - - > 2< 466944 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454807:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b049ba53 00000080:00200000:1.0:1787827240.454808:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 115. 00000080:00200000:1.0:1787827240.454809:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 115 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454818:0:48374:0:(rw26.c:875:ll_write_end()) pos 471040, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454821:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004c5caa56[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 115] 00000080:00200000:1.0:1787827240.454823:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d1f05e12 17ffffc0001001 3:0 ffff9a78870c9000 115 lru 00000080:00200000:1.0:1787827240.454827:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000678967c1 115: 1< 0 - - > 2< 471040 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454849:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004c5caa56 00000080:00200000:1.0:1787827240.454850:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 116. 00000080:00200000:1.0:1787827240.454852:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 116 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454864:0:48374:0:(rw26.c:875:ll_write_end()) pos 475136, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454868:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000dc01a49b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 116] 00000080:00200000:1.0:1787827240.454870:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006ddc3a98 17ffffc0001001 3:0 ffff9a78870c9cb0 116 lru 00000080:00200000:1.0:1787827240.454874:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a50124ee 116: 1< 0 - - > 2< 475136 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454876:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000dc01a49b 00000080:00200000:1.0:1787827240.454877:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 117. 00000080:00200000:1.0:1787827240.454878:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 117 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454887:0:48374:0:(rw26.c:875:ll_write_end()) pos 479232, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454890:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000499f8357[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 117] 00000080:00200000:1.0:1787827240.454892:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000053f5deec 17ffffc0001001 3:0 ffff9a78870c9ae0 117 lru 00000080:00200000:1.0:1787827240.454896:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c2639733 117: 1< 0 - - > 2< 479232 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454897:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000499f8357 00000080:00200000:1.0:1787827240.454898:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 118. 00000080:00200000:1.0:1787827240.454899:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 118 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454907:0:48374:0:(rw26.c:875:ll_write_end()) pos 483328, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454910:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004312bc0b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 118] 00000080:00200000:1.0:1787827240.454911:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005636531a 17ffffc0001001 3:0 ffff9a78870c9658 118 lru 00000080:00200000:1.0:1787827240.454915:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000041933d2f 118: 1< 0 - - > 2< 483328 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454916:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004312bc0b 00000080:00200000:1.0:1787827240.454917:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 119. 00000080:00200000:1.0:1787827240.454918:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 119 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454927:0:48374:0:(rw26.c:875:ll_write_end()) pos 487424, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454929:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000027b37452[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 119] 00000080:00200000:1.0:1787827240.454931:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a2d411d5 17ffffc0001001 3:0 ffff9a78870c99f8 119 lru 00000080:00200000:1.0:1787827240.454935:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f974f3a3 119: 1< 0 - - > 2< 487424 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454937:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000027b37452 00000080:00200000:1.0:1787827240.454938:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 120. 00000080:00200000:1.0:1787827240.454939:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 120 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454949:0:48374:0:(rw26.c:875:ll_write_end()) pos 491520, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454951:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008ff84fd0[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 120] 00000080:00200000:1.0:1787827240.454953:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006c2078d3 17ffffc0001001 3:0 ffff9a78870c9910 120 lru 00000080:00200000:1.0:1787827240.454957:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b25d9a43 120: 1< 0 - - > 2< 491520 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454959:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008ff84fd0 00000080:00200000:1.0:1787827240.454959:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 121. 00000080:00200000:1.0:1787827240.454961:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 121 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454969:0:48374:0:(rw26.c:875:ll_write_end()) pos 495616, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454972:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000024b820a1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 121] 00000080:00200000:1.0:1787827240.454973:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000393576c7 17ffffc0001001 3:0 ffff9a78870c9d98 121 lru 00000080:00200000:1.0:1787827240.454977:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b9e4fce0 121: 1< 0 - - > 2< 495616 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.454978:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000024b820a1 00000080:00200000:1.0:1787827240.454979:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 122. 00000080:00200000:1.0:1787827240.454980:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 122 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.454989:0:48374:0:(rw26.c:875:ll_write_end()) pos 499712, len 4096, copied 4096 00000080:00200000:1.0:1787827240.454992:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006149b411[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 122] 00000080:00200000:1.0:1787827240.454994:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001c4af120 17ffffc0001001 3:0 ffff9a78870c93a0 122 lru 00000080:00200000:1.0:1787827240.454998:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001ca24e13 122: 1< 0 - - > 2< 499712 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455000:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006149b411 00000080:00200000:1.0:1787827240.455000:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 123. 00000080:00200000:1.0:1787827240.455001:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 123 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455010:0:48374:0:(rw26.c:875:ll_write_end()) pos 503808, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455012:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f36bdce4[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 123] 00000080:00200000:1.0:1787827240.455014:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000cdbacebd 17ffffc0001001 3:0 ffff9a78870c90e8 123 lru 00000080:00200000:1.0:1787827240.455017:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004e8c87aa 123: 1< 0 - - > 2< 503808 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455019:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f36bdce4 00000080:00200000:1.0:1787827240.455019:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 124. 00000080:00200000:1.0:1787827240.455021:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 124 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455035:0:48374:0:(rw26.c:875:ll_write_end()) pos 507904, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455038:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000867745a9[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 124] 00000080:00200000:1.0:1787827240.455040:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000045d7e033 17ffffc0001001 3:0 ffff9a7884876000 124 lru 00000080:00200000:1.0:1787827240.455044:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000025f962f 124: 1< 0 - - > 2< 507904 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455046:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000867745a9 00000080:00200000:1.0:1787827240.455046:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 125. 00000080:00200000:1.0:1787827240.455047:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 125 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455056:0:48374:0:(rw26.c:875:ll_write_end()) pos 512000, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455059:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f9097421[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 125] 00000080:00200000:1.0:1787827240.455061:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000fe5d53e0 17ffffc0001001 3:0 ffff9a7884876cb0 125 lru 00000080:00200000:1.0:1787827240.455064:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000828b05ca 125: 1< 0 - - > 2< 512000 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455065:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f9097421 00000080:00200000:1.0:1787827240.455066:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 126. 00000080:00200000:1.0:1787827240.455068:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 126 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455076:0:48374:0:(rw26.c:875:ll_write_end()) pos 516096, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455079:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000174bd596[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 126] 00000080:00200000:1.0:1787827240.455081:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000f318cbb9 17ffffc0001001 3:0 ffff9a7884876ae0 126 lru 00000080:00200000:1.0:1787827240.455085:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000006b7d727d 126: 1< 0 - - > 2< 516096 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455086:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000174bd596 00000080:00200000:1.0:1787827240.455087:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 127. 00000080:00200000:1.0:1787827240.455088:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 127 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455098:0:48374:0:(rw26.c:875:ll_write_end()) pos 520192, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455100:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fd01d384[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 127] 00000080:00200000:1.0:1787827240.455102:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005b53ce7a 17ffffc0001001 3:0 ffff9a7884876658 127 lru 00000080:00200000:1.0:1787827240.455105:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000049880db0 127: 1< 0 - - > 2< 520192 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455107:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fd01d384 00000080:00200000:1.0:1787827240.455108:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 128. 00000080:00200000:1.0:1787827240.455109:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 128 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455120:0:48374:0:(rw26.c:875:ll_write_end()) pos 524288, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455123:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000019664dc7[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 128] 00000080:00200000:1.0:1787827240.455125:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008b68251f 17ffffc0001001 3:0 ffff9a78848769f8 128 lru 00000080:00200000:1.0:1787827240.455129:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000cc198900 128: 1< 0 - - > 2< 524288 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455131:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000019664dc7 00000080:00200000:1.0:1787827240.455132:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 129. 00000080:00200000:1.0:1787827240.455133:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 129 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455145:0:48374:0:(rw26.c:875:ll_write_end()) pos 528384, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455148:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d1f93d3c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 129] 00000080:00200000:1.0:1787827240.455150:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000071b2b151 17ffffc0001021 2:0 ffff9a7884876910 129 lru 00000080:00200000:1.0:1787827240.455154:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002a5f9f2e 129: 1< 0 - - > 2< 528384 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455155:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d1f93d3c 00000080:00200000:1.0:1787827240.455156:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 130. 00000080:00200000:1.0:1787827240.455157:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 130 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455165:0:48374:0:(rw26.c:875:ll_write_end()) pos 532480, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455168:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000a2765e88[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 130] 00000080:00200000:1.0:1787827240.455170:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000cd37dd3c 17ffffc0001001 3:0 ffff9a7884876d98 130 lru 00000080:00200000:1.0:1787827240.455173:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004e480d3b 130: 1< 0 - - > 2< 532480 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455174:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000a2765e88 00000080:00200000:1.0:1787827240.455175:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 131. 00000080:00200000:1.0:1787827240.455176:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 131 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455185:0:48374:0:(rw26.c:875:ll_write_end()) pos 536576, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455187:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000054bc82af[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 131] 00000080:00200000:1.0:1787827240.455189:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000000886bf26 17ffffc0001001 3:0 ffff9a78848763a0 131 lru 00000080:00200000:1.0:1787827240.455193:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000010feeb48 131: 1< 0 - - > 2< 536576 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455195:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000054bc82af 00000080:00200000:1.0:1787827240.455195:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 132. 00000080:00200000:1.0:1787827240.455197:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 132 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455205:0:48374:0:(rw26.c:875:ll_write_end()) pos 540672, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455208:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002e7c996b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 132] 00000080:00200000:1.0:1787827240.455209:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000cba60c32 17ffffc0001001 3:0 ffff9a78848760e8 132 lru 00000080:00200000:1.0:1787827240.455213:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004e02a7ee 132: 1< 0 - - > 2< 540672 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455214:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002e7c996b 00000080:00200000:1.0:1787827240.455215:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 133. 00000080:00200000:1.0:1787827240.455216:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 133 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455225:0:48374:0:(rw26.c:875:ll_write_end()) pos 544768, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455228:0:48374:0:(rw26.c:933:ll_write_end()) page@000000005a46a504[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 133] 00000080:00200000:1.0:1787827240.455230:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000094257cb3 17ffffc0001001 3:0 ffff9a78848762b8 133 lru 00000080:00200000:1.0:1787827240.455234:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000098473959 133: 1< 0 - - > 2< 544768 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455235:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000005a46a504 00000080:00200000:1.0:1787827240.455236:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 134. 00000080:00200000:1.0:1787827240.455237:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 134 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455247:0:48374:0:(rw26.c:875:ll_write_end()) pos 548864, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455249:0:48374:0:(rw26.c:933:ll_write_end()) page@000000000fcab6f9[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 134] 00000080:00200000:1.0:1787827240.455251:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d88cd58b 17ffffc0001001 3:0 ffff9a78848761d0 134 lru 00000080:00200000:1.0:1787827240.455255:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000082cb7e59 134: 1< 0 - - > 2< 548864 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455257:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000000fcab6f9 00000080:00200000:1.0:1787827240.455257:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 135. 00000080:00200000:1.0:1787827240.455259:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 135 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455267:0:48374:0:(rw26.c:875:ll_write_end()) pos 552960, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455270:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000705fdd27[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 135] 00000080:00200000:1.0:1787827240.455272:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d493cbf6 17ffffc0001001 3:0 ffff9a7884876740 135 lru 00000080:00200000:1.0:1787827240.455276:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000705b4054 135: 1< 0 - - > 2< 552960 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455277:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000705fdd27 00000080:00200000:1.0:1787827240.455278:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 136. 00000080:00200000:1.0:1787827240.455279:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 136 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455287:0:48374:0:(rw26.c:875:ll_write_end()) pos 557056, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455290:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000099e4641b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 136] 00000080:00200000:1.0:1787827240.455292:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006e3893e5 17ffffc0001001 3:0 ffff9a7884876e80 136 lru 00000080:00200000:1.0:1787827240.455295:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f02b0de4 136: 1< 0 - - > 2< 557056 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455297:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000099e4641b 00000080:00200000:1.0:1787827240.455298:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 137. 00000080:00200000:1.0:1787827240.455299:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 137 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455307:0:48374:0:(rw26.c:875:ll_write_end()) pos 561152, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455310:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002d9e45ea[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 137] 00000080:00200000:1.0:1787827240.455312:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000bad2bf3c 17ffffc0001001 3:0 ffff9a7884876488 137 lru 00000080:00200000:1.0:1787827240.455315:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004f111ea6 137: 1< 0 - - > 2< 561152 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455317:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002d9e45ea 00000080:00200000:1.0:1787827240.455318:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 138. 00000080:00200000:1.0:1787827240.455319:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 138 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455328:0:48374:0:(rw26.c:875:ll_write_end()) pos 565248, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455330:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000879ed984[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 138] 00000080:00200000:1.0:1787827240.455332:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006b0a9d47 17ffffc0001001 3:0 ffff9a7884876bc8 138 lru 00000080:00200000:1.0:1787827240.455336:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000109b1132 138: 1< 0 - - > 2< 565248 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455337:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000879ed984 00000080:00200000:1.0:1787827240.455339:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 139. 00000080:00200000:1.0:1787827240.455340:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 139 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455349:0:48374:0:(rw26.c:875:ll_write_end()) pos 569344, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455352:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000926c4c4a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 139] 00000080:00200000:1.0:1787827240.455354:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ad9d1496 17ffffc0001001 3:0 ffff9a7884876570 139 lru 00000080:00200000:1.0:1787827240.455358:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000008eec6bca 139: 1< 0 - - > 2< 569344 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455360:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000926c4c4a 00000080:00200000:1.0:1787827240.455361:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 140. 00000080:00200000:1.0:1787827240.455362:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 140 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455370:0:48374:0:(rw26.c:875:ll_write_end()) pos 573440, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455373:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000181bf031[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 140] 00000080:00200000:1.0:1787827240.455374:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000970b0ece 17ffffc0001001 3:0 ffff9a7884876828 140 lru 00000080:00200000:1.0:1787827240.455378:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004e2ee2d5 140: 1< 0 - - > 2< 573440 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455379:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000181bf031 00000080:00200000:1.0:1787827240.455380:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 141. 00000080:00200000:1.0:1787827240.455381:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 141 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455398:0:48374:0:(rw26.c:875:ll_write_end()) pos 577536, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455400:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000315ef5ab[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 141] 00000080:00200000:1.0:1787827240.455402:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009f75f597 17ffffc0001001 3:0 ffff9a7885a95570 141 lru 00000080:00200000:1.0:1787827240.455406:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e90980d7 141: 1< 0 - - > 2< 577536 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455408:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000315ef5ab 00000080:00200000:1.0:1787827240.455409:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 142. 00000080:00200000:1.0:1787827240.455410:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 142 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455419:0:48374:0:(rw26.c:875:ll_write_end()) pos 581632, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455422:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000849c676c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 142] 00000080:00200000:1.0:1787827240.455424:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000002896b855 17ffffc0001001 3:0 ffff9a7885a95828 142 lru 00000080:00200000:1.0:1787827240.455428:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a1dd0e23 142: 1< 0 - - > 2< 581632 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455430:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000849c676c 00000080:00200000:1.0:1787827240.455431:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 143. 00000080:00200000:1.0:1787827240.455432:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 143 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455440:0:48374:0:(rw26.c:875:ll_write_end()) pos 585728, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455443:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008d941ec6[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 143] 00000080:00200000:1.0:1787827240.455444:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004a7e5489 17ffffc0001001 3:0 ffff9a7885a95000 143 lru 00000080:00200000:1.0:1787827240.455448:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000d750b3e0 143: 1< 0 - - > 2< 585728 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455449:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008d941ec6 00000080:00200000:1.0:1787827240.455450:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 144. 00000080:00200000:1.0:1787827240.455452:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 144 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455463:0:48374:0:(rw26.c:875:ll_write_end()) pos 589824, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455465:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002c96488b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 144] 00000080:00200000:1.0:1787827240.455467:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d8b16dd9 17ffffc0001021 2:0 ffff9a7885a95cb0 144 lru 00000080:00200000:1.0:1787827240.455471:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000076df142d 144: 1< 0 - - > 2< 589824 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455473:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002c96488b 00000080:00200000:1.0:1787827240.455474:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 145. 00000080:00200000:1.0:1787827240.455475:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 145 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455484:0:48374:0:(rw26.c:875:ll_write_end()) pos 593920, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455486:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004777aaee[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 145] 00000080:00200000:1.0:1787827240.455488:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d407af5b 17ffffc0001001 3:0 ffff9a7885a95ae0 145 lru 00000080:00200000:1.0:1787827240.455492:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000053eb3490 145: 1< 0 - - > 2< 593920 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455493:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004777aaee 00000080:00200000:1.0:1787827240.455494:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 146. 00000080:00200000:1.0:1787827240.455495:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 146 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455504:0:48374:0:(rw26.c:875:ll_write_end()) pos 598016, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455506:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000da467a99[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 146] 00000080:00200000:1.0:1787827240.455521:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005c72a6cd 17ffffc0001001 3:0 ffff9a7885a95658 146 lru 00000080:00200000:1.0:1787827240.455525:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000078dba015 146: 1< 0 - - > 2< 598016 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455527:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000da467a99 00000080:00200000:1.0:1787827240.455528:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 147. 00000080:00200000:1.0:1787827240.455530:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 147 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455541:0:48374:0:(rw26.c:875:ll_write_end()) pos 602112, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455544:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000087d66ef2[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 147] 00000080:00200000:1.0:1787827240.455546:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000927dde2b 17ffffc0001001 3:0 ffff9a7885a959f8 147 lru 00000080:00200000:1.0:1787827240.455550:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000073dd05eb 147: 1< 0 - - > 2< 602112 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455551:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000087d66ef2 00000080:00200000:1.0:1787827240.455552:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 148. 00000080:00200000:1.0:1787827240.455553:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 148 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455563:0:48374:0:(rw26.c:875:ll_write_end()) pos 606208, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455566:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d10bf422[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 148] 00000080:00200000:1.0:1787827240.455567:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007835f9da 17ffffc0001001 3:0 ffff9a7885a95910 148 lru 00000080:00200000:1.0:1787827240.455571:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e7c4f2e0 148: 1< 0 - - > 2< 606208 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455572:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d10bf422 00000080:00200000:1.0:1787827240.455573:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 149. 00000080:00200000:1.0:1787827240.455574:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 149 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455583:0:48374:0:(rw26.c:875:ll_write_end()) pos 610304, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455585:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000464ee31e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 149] 00000080:00200000:1.0:1787827240.455587:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000525b30ea 17ffffc0001001 3:0 ffff9a7885a95d98 149 lru 00000080:00200000:1.0:1787827240.455591:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000cdd0a7c6 149: 1< 0 - - > 2< 610304 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455593:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000464ee31e 00000080:00200000:1.0:1787827240.455594:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 150. 00000080:00200000:1.0:1787827240.455595:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 150 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455604:0:48374:0:(rw26.c:875:ll_write_end()) pos 614400, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455606:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000097ec2758[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 150] 00000080:00200000:1.0:1787827240.455608:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000009df5ac0 17ffffc0001001 3:0 ffff9a7885a953a0 150 lru 00000080:00200000:1.0:1787827240.455611:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000da05b92a 150: 1< 0 - - > 2< 614400 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455613:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000097ec2758 00000080:00200000:1.0:1787827240.455614:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 151. 00000080:00200000:1.0:1787827240.455615:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 151 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455624:0:48374:0:(rw26.c:875:ll_write_end()) pos 618496, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455626:0:48374:0:(rw26.c:933:ll_write_end()) page@000000009bf02744[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 151] 00000080:00200000:1.0:1787827240.455629:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004ad9812b 17ffffc0001001 3:0 ffff9a7885a950e8 151 lru 00000080:00200000:1.0:1787827240.455632:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000006484f49f 151: 1< 0 - - > 2< 618496 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455634:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000009bf02744 00000080:00200000:1.0:1787827240.455635:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 152. 00000080:00200000:1.0:1787827240.455635:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 152 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455644:0:48374:0:(rw26.c:875:ll_write_end()) pos 622592, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455646:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002309baba[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 152] 00000080:00200000:1.0:1787827240.455648:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008f45f924 17ffffc0001001 3:0 ffff9a7885a952b8 152 lru 00000080:00200000:1.0:1787827240.455651:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a729175a 152: 1< 0 - - > 2< 622592 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455652:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002309baba 00000080:00200000:1.0:1787827240.455653:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 153. 00000080:00200000:1.0:1787827240.455655:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 153 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455663:0:48374:0:(rw26.c:875:ll_write_end()) pos 626688, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455665:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e9289a96[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 153] 00000080:00200000:1.0:1787827240.455668:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000747c7ca5 17ffffc0001001 3:0 ffff9a7885a951d0 153 lru 00000080:00200000:1.0:1787827240.455671:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c1493ee0 153: 1< 0 - - > 2< 626688 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455672:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e9289a96 00000080:00200000:1.0:1787827240.455673:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 154. 00000080:00200000:1.0:1787827240.455674:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 154 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455683:0:48374:0:(rw26.c:875:ll_write_end()) pos 630784, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455686:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004733915d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 154] 00000080:00200000:1.0:1787827240.455688:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000003a1098aa 17ffffc0001001 3:0 ffff9a7885a95740 154 lru 00000080:00200000:1.0:1787827240.455692:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c5c7564e 154: 1< 0 - - > 2< 630784 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455693:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004733915d 00000080:00200000:1.0:1787827240.455694:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 155. 00000080:00200000:1.0:1787827240.455695:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 155 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455705:0:48374:0:(rw26.c:875:ll_write_end()) pos 634880, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455707:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004a16da97[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 155] 00000080:00200000:1.0:1787827240.455709:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a68c9099 17ffffc0001001 3:0 ffff9a7885a95e80 155 lru 00000080:00200000:1.0:1787827240.455713:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002c3fddb3 155: 1< 0 - - > 2< 634880 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455715:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004a16da97 00000080:00200000:1.0:1787827240.455716:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 156. 00000080:00200000:1.0:1787827240.455717:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 156 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455725:0:48374:0:(rw26.c:875:ll_write_end()) pos 638976, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455728:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008bf4b9bf[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 156] 00000080:00200000:1.0:1787827240.455730:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000008a59a96 17ffffc0001001 3:0 ffff9a7885a95488 156 lru 00000080:00200000:1.0:1787827240.455734:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000007acc1490 156: 1< 0 - - > 2< 638976 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455735:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008bf4b9bf 00000080:00200000:1.0:1787827240.455736:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 157. 00000080:00200000:1.0:1787827240.455737:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 157 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455745:0:48374:0:(rw26.c:875:ll_write_end()) pos 643072, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455748:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ff2cc63a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 157] 00000080:00200000:1.0:1787827240.455749:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008211a9e0 17ffffc0001001 3:0 ffff9a7885a95bc8 157 lru 00000080:00200000:1.0:1787827240.455753:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004bb1e067 157: 1< 0 - - > 2< 643072 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455754:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ff2cc63a 00000080:00200000:1.0:1787827240.455755:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 158. 00000080:00200000:1.0:1787827240.455756:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 158 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455771:0:48374:0:(rw26.c:875:ll_write_end()) pos 647168, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455774:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004bdfc74e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 158] 00000080:00200000:1.0:1787827240.455776:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000000f669b 17ffffc0001001 3:0 ffff9a78887f3828 158 lru 00000080:00200000:1.0:1787827240.455780:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c3642a8c 158: 1< 0 - - > 2< 647168 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455781:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004bdfc74e 00000080:00200000:1.0:1787827240.455782:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 159. 00000080:00200000:1.0:1787827240.455783:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 159 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455794:0:48374:0:(rw26.c:875:ll_write_end()) pos 651264, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455797:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000044ef9fa4[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 159] 00000080:00200000:1.0:1787827240.455799:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000000e5739e1 17ffffc0001021 2:0 ffff9a78887f3000 159 lru 00000080:00200000:1.0:1787827240.455802:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000cca0843c 159: 1< 0 - - > 2< 651264 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455803:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000044ef9fa4 00000080:00200000:1.0:1787827240.455804:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 160. 00000080:00200000:1.0:1787827240.455805:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 160 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455814:0:48374:0:(rw26.c:875:ll_write_end()) pos 655360, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455817:0:48374:0:(rw26.c:933:ll_write_end()) page@000000009fe12a30[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 160] 00000080:00200000:1.0:1787827240.455819:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005b8722c1 17ffffc0001001 3:0 ffff9a78887f3cb0 160 lru 00000080:00200000:1.0:1787827240.455823:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000005cad3f3f 160: 1< 0 - - > 2< 655360 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455825:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000009fe12a30 00000080:00200000:1.0:1787827240.455825:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 161. 00000080:00200000:1.0:1787827240.455826:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 161 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455853:0:48374:0:(rw26.c:875:ll_write_end()) pos 659456, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455857:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008cb9ec40[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 161] 00000080:00200000:1.0:1787827240.455859:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000bbfbd6de 17ffffc0001001 3:0 ffff9a78887f3ae0 161 lru 00000080:00200000:1.0:1787827240.455863:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002f77bfef 161: 1< 0 - - > 2< 659456 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455865:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008cb9ec40 00000080:00200000:1.0:1787827240.455866:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 162. 00000080:00200000:1.0:1787827240.455868:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 162 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455879:0:48374:0:(rw26.c:875:ll_write_end()) pos 663552, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455883:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fd990648[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 162] 00000080:00200000:1.0:1787827240.455885:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004c7db868 17ffffc0001001 3:0 ffff9a78887f3658 162 lru 00000080:00200000:1.0:1787827240.455888:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000088d508b0 162: 1< 0 - - > 2< 663552 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455890:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fd990648 00000080:00200000:1.0:1787827240.455891:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 163. 00000080:00200000:1.0:1787827240.455892:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 163 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455901:0:48374:0:(rw26.c:875:ll_write_end()) pos 667648, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455904:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000753a3e3c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 163] 00000080:00200000:1.0:1787827240.455906:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a54a16f7 17ffffc0001001 3:0 ffff9a78887f39f8 163 lru 00000080:00200000:1.0:1787827240.455909:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000d9e63853 163: 1< 0 - - > 2< 667648 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455911:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000753a3e3c 00000080:00200000:1.0:1787827240.455912:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 164. 00000080:00200000:1.0:1787827240.455913:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 164 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455921:0:48374:0:(rw26.c:875:ll_write_end()) pos 671744, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455924:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d093dc20[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 164] 00000080:00200000:1.0:1787827240.455926:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004abb0055 17ffffc0001001 3:0 ffff9a78887f3910 164 lru 00000080:00200000:1.0:1787827240.455929:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000006926571a 164: 1< 0 - - > 2< 671744 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455931:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d093dc20 00000080:00200000:1.0:1787827240.455932:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 165. 00000080:00200000:1.0:1787827240.455933:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 165 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455941:0:48374:0:(rw26.c:875:ll_write_end()) pos 675840, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455944:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c1b8a40b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 165] 00000080:00200000:1.0:1787827240.455946:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000087316b55 17ffffc0001001 3:0 ffff9a78887f3d98 165 lru 00000080:00200000:1.0:1787827240.455950:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000cbb6f469 165: 1< 0 - - > 2< 675840 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455952:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c1b8a40b 00000080:00200000:1.0:1787827240.455952:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 166. 00000080:00200000:1.0:1787827240.455953:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 166 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455961:0:48374:0:(rw26.c:875:ll_write_end()) pos 679936, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455964:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000015c3cff8[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 166] 00000080:00200000:1.0:1787827240.455965:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000179513b7 17ffffc0001001 3:0 ffff9a78887f33a0 166 lru 00000080:00200000:1.0:1787827240.455969:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000093a8dd2b 166: 1< 0 - - > 2< 679936 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455970:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000015c3cff8 00000080:00200000:1.0:1787827240.455971:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 167. 00000080:00200000:1.0:1787827240.455972:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 167 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.455981:0:48374:0:(rw26.c:875:ll_write_end()) pos 684032, len 4096, copied 4096 00000080:00200000:1.0:1787827240.455983:0:48374:0:(rw26.c:933:ll_write_end()) page@000000003e40238d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 167] 00000080:00200000:1.0:1787827240.455985:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d421b279 17ffffc0001001 3:0 ffff9a78887f30e8 167 lru 00000080:00200000:1.0:1787827240.455989:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000026f7331c 167: 1< 0 - - > 2< 684032 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.455991:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000003e40238d 00000080:00200000:1.0:1787827240.455992:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 168. 00000080:00200000:1.0:1787827240.455993:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 168 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456001:0:48374:0:(rw26.c:875:ll_write_end()) pos 688128, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456004:0:48374:0:(rw26.c:933:ll_write_end()) page@000000001d5880b5[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 168] 00000080:00200000:1.0:1787827240.456006:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ec716e99 17ffffc0001001 3:0 ffff9a78887f32b8 168 lru 00000080:00200000:1.0:1787827240.456009:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000559e3546 168: 1< 0 - - > 2< 688128 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456011:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000001d5880b5 00000080:00200000:1.0:1787827240.456012:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 169. 00000080:00200000:1.0:1787827240.456013:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 169 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456023:0:48374:0:(rw26.c:875:ll_write_end()) pos 692224, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456026:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000de5c5740[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 169] 00000080:00200000:1.0:1787827240.456028:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009c56890d 17ffffc0001001 3:0 ffff9a78887f31d0 169 lru 00000080:00200000:1.0:1787827240.456032:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004e71279b 169: 1< 0 - - > 2< 692224 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456033:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000de5c5740 00000080:00200000:1.0:1787827240.456034:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 170. 00000080:00200000:1.0:1787827240.456035:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 170 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456043:0:48374:0:(rw26.c:875:ll_write_end()) pos 696320, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456046:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008d3b3d06[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 170] 00000080:00200000:1.0:1787827240.456047:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ac3f2ca0 17ffffc0001001 3:0 ffff9a78887f3740 170 lru 00000080:00200000:1.0:1787827240.456051:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000fc7ecd95 170: 1< 0 - - > 2< 696320 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456052:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008d3b3d06 00000080:00200000:1.0:1787827240.456053:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 171. 00000080:00200000:1.0:1787827240.456055:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 171 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456063:0:48374:0:(rw26.c:875:ll_write_end()) pos 700416, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456066:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004187d1b4[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 171] 00000080:00200000:1.0:1787827240.456068:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b836e427 17ffffc0001001 3:0 ffff9a78887f3e80 171 lru 00000080:00200000:1.0:1787827240.456072:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000094ca6e5b 171: 1< 0 - - > 2< 700416 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456073:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004187d1b4 00000080:00200000:1.0:1787827240.456074:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 172. 00000080:00200000:1.0:1787827240.456075:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 172 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456083:0:48374:0:(rw26.c:875:ll_write_end()) pos 704512, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456086:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ea8b5bf2[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 172] 00000080:00200000:1.0:1787827240.456087:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000058fd28a9 17ffffc0001001 3:0 ffff9a78887f3488 172 lru 00000080:00200000:1.0:1787827240.456091:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f4cc54de 172: 1< 0 - - > 2< 704512 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456092:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ea8b5bf2 00000080:00200000:1.0:1787827240.456093:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 173. 00000080:00200000:1.0:1787827240.456094:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 173 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456102:0:48374:0:(rw26.c:875:ll_write_end()) pos 708608, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456105:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000caa9710a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 173] 00000080:00200000:1.0:1787827240.456107:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000097d51aad 17ffffc0001001 3:0 ffff9a78887f3bc8 173 lru 00000080:00200000:1.0:1787827240.456111:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000096d15547 173: 1< 0 - - > 2< 708608 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456113:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000caa9710a 00000080:00200000:1.0:1787827240.456114:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 174. 00000080:00200000:1.0:1787827240.456115:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 174 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456126:0:48374:0:(rw26.c:875:ll_write_end()) pos 712704, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456129:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000034a1414b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 174] 00000080:00200000:1.0:1787827240.456131:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000016da9dd5 17ffffc0001021 2:0 ffff9a78887f3570 174 lru 00000080:00200000:1.0:1787827240.456135:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000305ba6e4 174: 1< 0 - - > 2< 712704 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456136:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000034a1414b 00000080:00200000:1.0:1787827240.456137:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 175. 00000080:00200000:1.0:1787827240.456138:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 175 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456155:0:48374:0:(rw26.c:875:ll_write_end()) pos 716800, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456158:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c9f5587d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 175] 00000080:00200000:1.0:1787827240.456160:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000094d56ca7 17ffffc0001001 3:0 ffff9a78884643a0 175 lru 00000080:00200000:1.0:1787827240.456163:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000904b7736 175: 1< 0 - - > 2< 716800 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456164:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c9f5587d 00000080:00200000:1.0:1787827240.456165:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 176. 00000080:00200000:1.0:1787827240.456166:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 176 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456176:0:48374:0:(rw26.c:875:ll_write_end()) pos 720896, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456179:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ab98d02e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 176] 00000080:00200000:1.0:1787827240.456181:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008ec98420 17ffffc0001001 3:0 ffff9a78884640e8 176 lru 00000080:00200000:1.0:1787827240.456184:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f3ca5d9c 176: 1< 0 - - > 2< 720896 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456186:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ab98d02e 00000080:00200000:1.0:1787827240.456187:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 177. 00000080:00200000:1.0:1787827240.456188:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 177 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456196:0:48374:0:(rw26.c:875:ll_write_end()) pos 724992, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456199:0:48374:0:(rw26.c:933:ll_write_end()) page@000000003ee009af[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 177] 00000080:00200000:1.0:1787827240.456200:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000070d3d9df 17ffffc0001001 3:0 ffff9a78884642b8 177 lru 00000080:00200000:1.0:1787827240.456204:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000dfe44eb8 177: 1< 0 - - > 2< 724992 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456205:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000003ee009af 00000080:00200000:1.0:1787827240.456206:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 178. 00000080:00200000:1.0:1787827240.456207:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 178 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456216:0:48374:0:(rw26.c:875:ll_write_end()) pos 729088, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456218:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000029136a25[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 178] 00000080:00200000:1.0:1787827240.456220:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e073b353 17ffffc0001001 3:0 ffff9a78884641d0 178 lru 00000080:00200000:1.0:1787827240.456224:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000826a1518 178: 1< 0 - - > 2< 729088 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456225:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000029136a25 00000080:00200000:1.0:1787827240.456226:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 179. 00000080:00200000:1.0:1787827240.456228:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 179 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456236:0:48374:0:(rw26.c:875:ll_write_end()) pos 733184, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456239:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000de4f6ddb[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 179] 00000080:00200000:1.0:1787827240.456241:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b4723ef1 17ffffc0001001 3:0 ffff9a7888464740 179 lru 00000080:00200000:1.0:1787827240.456245:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001cab8bbc 179: 1< 0 - - > 2< 733184 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456247:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000de4f6ddb 00000080:00200000:1.0:1787827240.456247:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 180. 00000080:00200000:1.0:1787827240.456248:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 180 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456257:0:48374:0:(rw26.c:875:ll_write_end()) pos 737280, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456259:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000196857d3[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 180] 00000080:00200000:1.0:1787827240.456261:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000056f22a14 17ffffc0001001 3:0 ffff9a7888464e80 180 lru 00000080:00200000:1.0:1787827240.456265:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000005dcd8fb7 180: 1< 0 - - > 2< 737280 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456266:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000196857d3 00000080:00200000:1.0:1787827240.456267:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 181. 00000080:00200000:1.0:1787827240.456268:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 181 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456277:0:48374:0:(rw26.c:875:ll_write_end()) pos 741376, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456279:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006300fc08[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 181] 00000080:00200000:1.0:1787827240.456281:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001dd81042 17ffffc0001001 3:0 ffff9a7888464488 181 lru 00000080:00200000:1.0:1787827240.456285:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000ab25277 181: 1< 0 - - > 2< 741376 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456286:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006300fc08 00000080:00200000:1.0:1787827240.456286:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 182. 00000080:00200000:1.0:1787827240.456288:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 182 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456296:0:48374:0:(rw26.c:875:ll_write_end()) pos 745472, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456299:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008caa771d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 182] 00000080:00200000:1.0:1787827240.456301:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006fc8d52f 17ffffc0001001 3:0 ffff9a7888464bc8 182 lru 00000080:00200000:1.0:1787827240.456305:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a4a602fe 182: 1< 0 - - > 2< 745472 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456307:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008caa771d 00000080:00200000:1.0:1787827240.456308:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 183. 00000080:00200000:1.0:1787827240.456309:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 183 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456318:0:48374:0:(rw26.c:875:ll_write_end()) pos 749568, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456321:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b1c384c6[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 183] 00000080:00200000:1.0:1787827240.456323:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000012baa58c 17ffffc0001001 3:0 ffff9a7888464570 183 lru 00000080:00200000:1.0:1787827240.456327:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e1d1f474 183: 1< 0 - - > 2< 749568 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456328:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b1c384c6 00000080:00200000:1.0:1787827240.456329:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 184. 00000080:00200000:1.0:1787827240.456330:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 184 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456338:0:48374:0:(rw26.c:875:ll_write_end()) pos 753664, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456340:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000039fb7669[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 184] 00000080:00200000:1.0:1787827240.456342:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000922e3a21 17ffffc0001001 3:0 ffff9a7888464828 184 lru 00000080:00200000:1.0:1787827240.456346:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000537eeee4 184: 1< 0 - - > 2< 753664 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456347:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000039fb7669 00000080:00200000:1.0:1787827240.456348:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 185. 00000080:00200000:1.0:1787827240.456349:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 185 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456358:0:48374:0:(rw26.c:875:ll_write_end()) pos 757760, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456361:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000859042f6[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 185] 00000080:00200000:1.0:1787827240.456363:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c3dc4dcc 17ffffc0001001 3:0 ffff9a7888464000 185 lru 00000080:00200000:1.0:1787827240.456367:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000008ba867b9 185: 1< 0 - - > 2< 757760 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456368:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000859042f6 00000080:00200000:1.0:1787827240.456369:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 186. 00000080:00200000:1.0:1787827240.456370:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 186 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456378:0:48374:0:(rw26.c:875:ll_write_end()) pos 761856, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456381:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000041d67744[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 186] 00000080:00200000:1.0:1787827240.456383:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000dc602946 17ffffc0001001 3:0 ffff9a7888464cb0 186 lru 00000080:00200000:1.0:1787827240.456386:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001963510f 186: 1< 0 - - > 2< 761856 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456388:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000041d67744 00000080:00200000:1.0:1787827240.456389:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 187. 00000080:00200000:1.0:1787827240.456390:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 187 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456399:0:48374:0:(rw26.c:875:ll_write_end()) pos 765952, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456402:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000095dec199[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 187] 00000080:00200000:1.0:1787827240.456404:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000525a1e73 17ffffc0001001 3:0 ffff9a7888464ae0 187 lru 00000080:00200000:1.0:1787827240.456408:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b9a57eb1 187: 1< 0 - - > 2< 765952 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456409:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000095dec199 00000080:00200000:1.0:1787827240.456410:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 188. 00000080:00200000:1.0:1787827240.456411:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 188 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456419:0:48374:0:(rw26.c:875:ll_write_end()) pos 770048, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456422:0:48374:0:(rw26.c:933:ll_write_end()) page@000000007091f8f4[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 188] 00000080:00200000:1.0:1787827240.456424:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009d81b052 17ffffc0001001 3:0 ffff9a7888464658 188 lru 00000080:00200000:1.0:1787827240.456427:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000007556ea9 188: 1< 0 - - > 2< 770048 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456428:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000007091f8f4 00000080:00200000:1.0:1787827240.456429:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 189. 00000080:00200000:1.0:1787827240.456430:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 189 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456440:0:48374:0:(rw26.c:875:ll_write_end()) pos 774144, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456443:0:48374:0:(rw26.c:933:ll_write_end()) page@000000005a97ae35[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 189] 00000080:00200000:1.0:1787827240.456445:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e9f49b53 17ffffc0001021 2:0 ffff9a78884649f8 189 lru 00000080:00200000:1.0:1787827240.456448:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e2d418b1 189: 1< 0 - - > 2< 774144 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456450:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000005a97ae35 00000080:00200000:1.0:1787827240.456450:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 190. 00000080:00200000:1.0:1787827240.456451:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 190 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456461:0:48374:0:(rw26.c:875:ll_write_end()) pos 778240, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456463:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000c2184626[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 190] 00000080:00200000:1.0:1787827240.456465:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000003d2478bb 17ffffc0001001 3:0 ffff9a7888464910 190 lru 00000080:00200000:1.0:1787827240.456469:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000ac93c182 190: 1< 0 - - > 2< 778240 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456471:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000c2184626 00000080:00200000:1.0:1787827240.456471:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 191. 00000080:00200000:1.0:1787827240.456473:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 191 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456481:0:48374:0:(rw26.c:875:ll_write_end()) pos 782336, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456484:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006145142f[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 191] 00000080:00200000:1.0:1787827240.456485:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000091f84bd2 17ffffc0001001 3:0 ffff9a7888464d98 191 lru 00000080:00200000:1.0:1787827240.456489:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000fd59f275 191: 1< 0 - - > 2< 782336 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456491:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006145142f 00000080:00200000:1.0:1787827240.456492:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 192. 00000080:00200000:1.0:1787827240.456493:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 192 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456523:0:48374:0:(rw26.c:875:ll_write_end()) pos 786432, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456526:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000e8b2b842[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 192] 00000080:00200000:1.0:1787827240.456528:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000076eae0b2 17ffffc0001001 3:0 ffff9a789092e1d0 192 lru 00000080:00200000:1.0:1787827240.456532:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e7613c13 192: 1< 0 - - > 2< 786432 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456534:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000e8b2b842 00000080:00200000:1.0:1787827240.456535:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 193. 00000080:00200000:1.0:1787827240.456536:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 193 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456547:0:48374:0:(rw26.c:875:ll_write_end()) pos 790528, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456549:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006034ae4a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 193] 00000080:00200000:1.0:1787827240.456551:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000affe9c2d 17ffffc0001001 3:0 ffff9a789092e740 193 lru 00000080:00200000:1.0:1787827240.456555:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000ddf02f2 193: 1< 0 - - > 2< 790528 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456557:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006034ae4a 00000080:00200000:1.0:1787827240.456557:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 194. 00000080:00200000:1.0:1787827240.456559:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 194 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456567:0:48374:0:(rw26.c:875:ll_write_end()) pos 794624, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456569:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000891e7988[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 194] 00000080:00200000:1.0:1787827240.456571:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d6b1fa2f 17ffffc0001001 3:0 ffff9a789092ee80 194 lru 00000080:00200000:1.0:1787827240.456575:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000957f7a72 194: 1< 0 - - > 2< 794624 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456577:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000891e7988 00000080:00200000:1.0:1787827240.456578:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 195. 00000080:00200000:1.0:1787827240.456579:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 195 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456587:0:48374:0:(rw26.c:875:ll_write_end()) pos 798720, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456590:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000288fa8bb[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 195] 00000080:00200000:1.0:1787827240.456592:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007140a062 17ffffc0001001 3:0 ffff9a789092e488 195 lru 00000080:00200000:1.0:1787827240.456595:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000174b951d 195: 1< 0 - - > 2< 798720 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456597:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000288fa8bb 00000080:00200000:1.0:1787827240.456598:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 196. 00000080:00200000:1.0:1787827240.456599:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 196 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456608:0:48374:0:(rw26.c:875:ll_write_end()) pos 802816, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456611:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b5245437[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 196] 00000080:00200000:1.0:1787827240.456612:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000020537c1 17ffffc0001001 3:0 ffff9a789092ebc8 196 lru 00000080:00200000:1.0:1787827240.456616:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c021d656 196: 1< 0 - - > 2< 802816 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456618:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b5245437 00000080:00200000:1.0:1787827240.456618:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 197. 00000080:00200000:1.0:1787827240.456620:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 197 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456630:0:48374:0:(rw26.c:875:ll_write_end()) pos 806912, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456633:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f10c56d3[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 197] 00000080:00200000:1.0:1787827240.456635:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000497e4e66 17ffffc0001001 3:0 ffff9a789092e570 197 lru 00000080:00200000:1.0:1787827240.456639:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000070c6dad2 197: 1< 0 - - > 2< 806912 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456640:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f10c56d3 00000080:00200000:1.0:1787827240.456641:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 198. 00000080:00200000:1.0:1787827240.456642:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 198 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456651:0:48374:0:(rw26.c:875:ll_write_end()) pos 811008, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456653:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000dde1e845[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 198] 00000080:00200000:1.0:1787827240.456655:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b3606bf2 17ffffc0001001 3:0 ffff9a789092e828 198 lru 00000080:00200000:1.0:1787827240.456658:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000549f287 198: 1< 0 - - > 2< 811008 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456660:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000dde1e845 00000080:00200000:1.0:1787827240.456660:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 199. 00000080:00200000:1.0:1787827240.456662:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 199 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456671:0:48374:0:(rw26.c:875:ll_write_end()) pos 815104, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456674:0:48374:0:(rw26.c:933:ll_write_end()) page@000000005db7403b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 199] 00000080:00200000:1.0:1787827240.456675:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e55cb578 17ffffc0001001 3:0 ffff9a789092e000 199 lru 00000080:00200000:1.0:1787827240.456679:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b61cf52d 199: 1< 0 - - > 2< 815104 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456680:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000005db7403b 00000080:00200000:1.0:1787827240.456681:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 200. 00000080:00200000:1.0:1787827240.456682:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 200 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456690:0:48374:0:(rw26.c:875:ll_write_end()) pos 819200, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456692:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000724d3071[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 200] 00000080:00200000:1.0:1787827240.456694:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000294bfcfc 17ffffc0001001 3:0 ffff9a789092ecb0 200 lru 00000080:00200000:1.0:1787827240.456697:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000043c64615 200: 1< 0 - - > 2< 819200 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456699:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000724d3071 00000080:00200000:1.0:1787827240.456700:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 201. 00000080:00200000:1.0:1787827240.456701:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 201 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456710:0:48374:0:(rw26.c:875:ll_write_end()) pos 823296, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456713:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b55450d1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 201] 00000080:00200000:1.0:1787827240.456715:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001fefc2b6 17ffffc0001001 3:0 ffff9a789092eae0 201 lru 00000080:00200000:1.0:1787827240.456718:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000072c85949 201: 1< 0 - - > 2< 823296 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456720:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b55450d1 00000080:00200000:1.0:1787827240.456721:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 202. 00000080:00200000:1.0:1787827240.456722:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 202 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456730:0:48374:0:(rw26.c:875:ll_write_end()) pos 827392, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456732:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000127e6d9d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 202] 00000080:00200000:1.0:1787827240.456734:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009c292948 17ffffc0001001 3:0 ffff9a789092e658 202 lru 00000080:00200000:1.0:1787827240.456737:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a4ecebad 202: 1< 0 - - > 2< 827392 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456738:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000127e6d9d 00000080:00200000:1.0:1787827240.456740:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 203. 00000080:00200000:1.0:1787827240.456741:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 203 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456755:0:48374:0:(rw26.c:875:ll_write_end()) pos 831488, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456759:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000057749d86[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 203] 00000080:00200000:1.0:1787827240.456763:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000083f04eeb 17ffffc0001001 3:0 ffff9a789092e9f8 203 lru 00000080:00200000:1.0:1787827240.456770:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000009e9099b7 203: 1< 0 - - > 2< 831488 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456772:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000057749d86 00000080:00200000:1.0:1787827240.456774:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 204. 00000080:00200000:1.0:1787827240.456776:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 204 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456797:0:48374:0:(rw26.c:875:ll_write_end()) pos 835584, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456802:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000af363020[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 204] 00000080:00200000:1.0:1787827240.456805:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005ae1bc22 17ffffc0001021 2:0 ffff9a789092e910 204 lru 00000080:00200000:1.0:1787827240.456810:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000006a1fb4f5 204: 1< 0 - - > 2< 835584 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456812:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000af363020 00000080:00200000:1.0:1787827240.456814:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 205. 00000080:00200000:1.0:1787827240.456816:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 205 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.456963:0:48374:0:(rw26.c:875:ll_write_end()) pos 839680, len 4096, copied 4096 00000080:00200000:1.0:1787827240.456967:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ff840e04[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 205] 00000080:00200000:1.0:1787827240.456969:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a35c47b2 17ffffc0001001 3:0 ffff9a789092ed98 205 lru 00000080:00200000:1.0:1787827240.456974:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000058c63815 205: 1< 0 - - > 2< 839680 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.456976:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ff840e04 00000080:00200000:1.0:1787827240.456976:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 206. 00000080:00200000:1.0:1787827240.457172:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 206 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457187:0:48374:0:(rw26.c:875:ll_write_end()) pos 843776, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457190:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fe211c8e[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 206] 00000080:00200000:1.0:1787827240.457192:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000b5cc704b 17ffffc0001001 3:0 ffff9a789092e3a0 206 lru 00000080:00200000:1.0:1787827240.457196:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000000c82b25 206: 1< 0 - - > 2< 843776 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457198:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fe211c8e 00000080:00200000:1.0:1787827240.457199:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 207. 00000080:00200000:1.0:1787827240.457200:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 207 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457209:0:48374:0:(rw26.c:875:ll_write_end()) pos 847872, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457212:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ef77751d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 207] 00000080:00200000:1.0:1787827240.457213:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d3c3950a 17ffffc0001001 3:0 ffff9a789092e0e8 207 lru 00000080:00200000:1.0:1787827240.457217:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c11a9290 207: 1< 0 - - > 2< 847872 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457219:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ef77751d 00000080:00200000:1.0:1787827240.457220:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 208. 00000080:00200000:1.0:1787827240.457221:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 208 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457229:0:48374:0:(rw26.c:875:ll_write_end()) pos 851968, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457232:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000042a87fa3[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 208] 00000080:00200000:1.0:1787827240.457234:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000064b3278d 17ffffc0001001 3:0 ffff9a789092e2b8 208 lru 00000080:00200000:1.0:1787827240.457238:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f53c6b52 208: 1< 0 - - > 2< 851968 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457239:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000042a87fa3 00000080:00200000:1.0:1787827240.457240:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 209. 00000080:00200000:1.0:1787827240.457241:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 209 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457256:0:48374:0:(rw26.c:875:ll_write_end()) pos 856064, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457259:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000057aca4a5[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 209] 00000080:00200000:1.0:1787827240.457261:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ada1bcff 17ffffc0001001 3:0 ffff9a7888600cb0 209 lru 00000080:00200000:1.0:1787827240.457265:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000847199f7 209: 1< 0 - - > 2< 856064 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457267:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000057aca4a5 00000080:00200000:1.0:1787827240.457268:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 210. 00000080:00200000:1.0:1787827240.457269:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 210 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457278:0:48374:0:(rw26.c:875:ll_write_end()) pos 860160, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457280:0:48374:0:(rw26.c:933:ll_write_end()) page@000000007f600408[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 210] 00000080:00200000:1.0:1787827240.457282:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004418fb64 17ffffc0001001 3:0 ffff9a7888600ae0 210 lru 00000080:00200000:1.0:1787827240.457286:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a6c232aa 210: 1< 0 - - > 2< 860160 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457287:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000007f600408 00000080:00200000:1.0:1787827240.457288:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 211. 00000080:00200000:1.0:1787827240.457290:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 211 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457299:0:48374:0:(rw26.c:875:ll_write_end()) pos 864256, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457301:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008eaf9134[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 211] 00000080:00200000:1.0:1787827240.457303:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000017c441c0 17ffffc0001001 3:0 ffff9a7888600658 211 lru 00000080:00200000:1.0:1787827240.457307:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000cdc8be04 211: 1< 0 - - > 2< 864256 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457309:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008eaf9134 00000080:00200000:1.0:1787827240.457310:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 212. 00000080:00200000:1.0:1787827240.457311:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 212 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457319:0:48374:0:(rw26.c:875:ll_write_end()) pos 868352, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457321:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000011a7e1fc[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 212] 00000080:00200000:1.0:1787827240.457323:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000000bd4277e 17ffffc0001001 3:0 ffff9a78886009f8 212 lru 00000080:00200000:1.0:1787827240.457326:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000aa9fdec5 212: 1< 0 - - > 2< 868352 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457328:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000011a7e1fc 00000080:00200000:1.0:1787827240.457328:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 213. 00000080:00200000:1.0:1787827240.457329:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 213 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457339:0:48374:0:(rw26.c:875:ll_write_end()) pos 872448, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457342:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000f882088c[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 213] 00000080:00200000:1.0:1787827240.457343:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d6b9917a 17ffffc0001001 3:0 ffff9a7888600910 213 lru 00000080:00200000:1.0:1787827240.457347:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000007b1f8dde 213: 1< 0 - - > 2< 872448 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457348:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000f882088c 00000080:00200000:1.0:1787827240.457348:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 214. 00000080:00200000:1.0:1787827240.457350:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 214 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457358:0:48374:0:(rw26.c:875:ll_write_end()) pos 876544, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457361:0:48374:0:(rw26.c:933:ll_write_end()) page@000000007f768e45[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 214] 00000080:00200000:1.0:1787827240.457363:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001c6027c8 17ffffc0001001 3:0 ffff9a7888600d98 214 lru 00000080:00200000:1.0:1787827240.457366:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c4604ab9 214: 1< 0 - - > 2< 876544 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457368:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000007f768e45 00000080:00200000:1.0:1787827240.457369:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 215. 00000080:00200000:1.0:1787827240.457370:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 215 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457378:0:48374:0:(rw26.c:875:ll_write_end()) pos 880640, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457381:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000081551c44[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 215] 00000080:00200000:1.0:1787827240.457382:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d0116f0c 17ffffc0001001 3:0 ffff9a78886003a0 215 lru 00000080:00200000:1.0:1787827240.457386:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e1ac78b3 215: 1< 0 - - > 2< 880640 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457387:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000081551c44 00000080:00200000:1.0:1787827240.457388:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 216. 00000080:00200000:1.0:1787827240.457389:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 216 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457398:0:48374:0:(rw26.c:875:ll_write_end()) pos 884736, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457401:0:48374:0:(rw26.c:933:ll_write_end()) page@000000004131ec7d[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 216] 00000080:00200000:1.0:1787827240.457403:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000078d9bbd0 17ffffc0001001 3:0 ffff9a78886000e8 216 lru 00000080:00200000:1.0:1787827240.457407:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000fa5b3837 216: 1< 0 - - > 2< 884736 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457409:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000004131ec7d 00000080:00200000:1.0:1787827240.457409:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 217. 00000080:00200000:1.0:1787827240.457410:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 217 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457418:0:48374:0:(rw26.c:875:ll_write_end()) pos 888832, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457421:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000da7875ce[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 217] 00000080:00200000:1.0:1787827240.457423:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000edf3748e 17ffffc0001001 3:0 ffff9a78886002b8 217 lru 00000080:00200000:1.0:1787827240.457426:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000073b0fbbc 217: 1< 0 - - > 2< 888832 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457428:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000da7875ce 00000080:00200000:1.0:1787827240.457428:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 218. 00000080:00200000:1.0:1787827240.457430:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 218 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457438:0:48374:0:(rw26.c:875:ll_write_end()) pos 892928, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457440:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fab9cdec[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 218] 00000080:00200000:1.0:1787827240.457442:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000006bf8382a 17ffffc0001001 3:0 ffff9a78886001d0 218 lru 00000080:00200000:1.0:1787827240.457447:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000094828e30 218: 1< 0 - - > 2< 892928 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457448:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fab9cdec 00000080:00200000:1.0:1787827240.457449:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 219. 00000080:00200000:1.0:1787827240.457451:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 219 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457462:0:48374:0:(rw26.c:875:ll_write_end()) pos 897024, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457465:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000001fe5b12[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 219] 00000080:00200000:1.0:1787827240.457467:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000bfa206ed 17ffffc0001021 2:0 ffff9a7888600740 219 lru 00000080:00200000:1.0:1787827240.457471:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e0c51db1 219: 1< 0 - - > 2< 897024 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457472:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000001fe5b12 00000080:00200000:1.0:1787827240.457473:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 220. 00000080:00200000:1.0:1787827240.457474:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 220 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457483:0:48374:0:(rw26.c:875:ll_write_end()) pos 901120, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457486:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000237897d0[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 220] 00000080:00200000:1.0:1787827240.457487:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000028f73146 17ffffc0001001 3:0 ffff9a7888600e80 220 lru 00000080:00200000:1.0:1787827240.457491:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b0e64064 220: 1< 0 - - > 2< 901120 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457492:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000237897d0 00000080:00200000:1.0:1787827240.457493:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 221. 00000080:00200000:1.0:1787827240.457494:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 221 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457503:0:48374:0:(rw26.c:875:ll_write_end()) pos 905216, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457506:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000986a7779[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 221] 00000080:00200000:1.0:1787827240.457520:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ed2e5765 17ffffc0001001 3:0 ffff9a7888600488 221 lru 00000080:00200000:1.0:1787827240.457526:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000a245522d 221: 1< 0 - - > 2< 905216 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457527:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000986a7779 00000080:00200000:1.0:1787827240.457528:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 222. 00000080:00200000:1.0:1787827240.457530:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 222 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457542:0:48374:0:(rw26.c:875:ll_write_end()) pos 909312, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457544:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000b0048f60[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 222] 00000080:00200000:1.0:1787827240.457546:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000064de49f6 17ffffc0001001 3:0 ffff9a7888600bc8 222 lru 00000080:00200000:1.0:1787827240.457550:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000f13c0f0c 222: 1< 0 - - > 2< 909312 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457552:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000b0048f60 00000080:00200000:1.0:1787827240.457553:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 223. 00000080:00200000:1.0:1787827240.457554:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 223 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457563:0:48374:0:(rw26.c:875:ll_write_end()) pos 913408, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457566:0:48374:0:(rw26.c:933:ll_write_end()) page@000000008a57a513[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 223] 00000080:00200000:1.0:1787827240.457568:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000853aa523 17ffffc0001001 3:0 ffff9a7888600570 223 lru 00000080:00200000:1.0:1787827240.457572:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000de50822e 223: 1< 0 - - > 2< 913408 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457573:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000008a57a513 00000080:00200000:1.0:1787827240.457574:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 224. 00000080:00200000:1.0:1787827240.457575:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 224 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457583:0:48374:0:(rw26.c:875:ll_write_end()) pos 917504, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457586:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000089354c46[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 224] 00000080:00200000:1.0:1787827240.457588:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ad4fdf74 17ffffc0001001 3:0 ffff9a7888600828 224 lru 00000080:00200000:1.0:1787827240.457591:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000004095bb89 224: 1< 0 - - > 2< 917504 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457593:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000089354c46 00000080:00200000:1.0:1787827240.457593:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 225. 00000080:00200000:1.0:1787827240.457595:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 225 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457603:0:48374:0:(rw26.c:875:ll_write_end()) pos 921600, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457606:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000948e443a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 225] 00000080:00200000:1.0:1787827240.457607:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000069ff5fbe 17ffffc0001001 3:0 ffff9a7888600000 225 lru 00000080:00200000:1.0:1787827240.457611:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003bb9dd1e 225: 1< 0 - - > 2< 921600 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457612:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000948e443a 00000080:00200000:1.0:1787827240.457613:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 226. 00000080:00200000:1.0:1787827240.457614:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 226 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457629:0:48374:0:(rw26.c:875:ll_write_end()) pos 925696, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457632:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000080a17d10[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 226] 00000080:00200000:1.0:1787827240.457633:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000072d901de 17ffffc0001001 3:0 ffff9a78847870e8 226 lru 00000080:00200000:1.0:1787827240.457637:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000008dad8f7a 226: 1< 0 - - > 2< 925696 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457638:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000080a17d10 00000080:00200000:1.0:1787827240.457639:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 227. 00000080:00200000:1.0:1787827240.457640:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 227 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457650:0:48374:0:(rw26.c:875:ll_write_end()) pos 929792, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457653:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000096a28fc8[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 227] 00000080:00200000:1.0:1787827240.457655:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000009cd3710b 17ffffc0001001 3:0 ffff9a78847872b8 227 lru 00000080:00200000:1.0:1787827240.457659:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000001f11e639 227: 1< 0 - - > 2< 929792 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457661:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000096a28fc8 00000080:00200000:1.0:1787827240.457662:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 228. 00000080:00200000:1.0:1787827240.457663:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 228 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457672:0:48374:0:(rw26.c:875:ll_write_end()) pos 933888, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457674:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d85fd132[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 228] 00000080:00200000:1.0:1787827240.457676:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000002ee13e12 17ffffc0001001 3:0 ffff9a78847871d0 228 lru 00000080:00200000:1.0:1787827240.457680:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b819ec53 228: 1< 0 - - > 2< 933888 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457682:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d85fd132 00000080:00200000:1.0:1787827240.457683:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 229. 00000080:00200000:1.0:1787827240.457684:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 229 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457693:0:48374:0:(rw26.c:875:ll_write_end()) pos 937984, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457695:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000090c84ae8[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 229] 00000080:00200000:1.0:1787827240.457697:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000008da3df5c 17ffffc0001001 3:0 ffff9a7884787740 229 lru 00000080:00200000:1.0:1787827240.457700:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000098e9019f 229: 1< 0 - - > 2< 937984 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457702:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000090c84ae8 00000080:00200000:1.0:1787827240.457702:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 230. 00000080:00200000:1.0:1787827240.457704:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 230 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457712:0:48374:0:(rw26.c:875:ll_write_end()) pos 942080, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457715:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000571ecf18[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 230] 00000080:00200000:1.0:1787827240.457717:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000d8ca96ff 17ffffc0001001 3:0 ffff9a7884787e80 230 lru 00000080:00200000:1.0:1787827240.457721:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000023ca4dc2 230: 1< 0 - - > 2< 942080 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457722:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000571ecf18 00000080:00200000:1.0:1787827240.457723:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 231. 00000080:00200000:1.0:1787827240.457724:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 231 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457733:0:48374:0:(rw26.c:875:ll_write_end()) pos 946176, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457735:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000ace8ae93[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 231] 00000080:00200000:1.0:1787827240.457737:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000005440bf8b 17ffffc0001001 3:0 ffff9a7884787488 231 lru 00000080:00200000:1.0:1787827240.457740:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000003bdf9af 231: 1< 0 - - > 2< 946176 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457742:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000ace8ae93 00000080:00200000:1.0:1787827240.457743:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 232. 00000080:00200000:1.0:1787827240.457744:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 232 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457753:0:48374:0:(rw26.c:875:ll_write_end()) pos 950272, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457755:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000a80107bf[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 232] 00000080:00200000:1.0:1787827240.457757:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000057105d06 17ffffc0001001 3:0 ffff9a7884787bc8 232 lru 00000080:00200000:1.0:1787827240.457761:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000012f62530 232: 1< 0 - - > 2< 950272 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457762:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000a80107bf 00000080:00200000:1.0:1787827240.457763:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 233. 00000080:00200000:1.0:1787827240.457764:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 233 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457772:0:48374:0:(rw26.c:875:ll_write_end()) pos 954368, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457775:0:48374:0:(rw26.c:933:ll_write_end()) page@000000001662884b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 233] 00000080:00200000:1.0:1787827240.457776:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000c04d320a 17ffffc0001001 3:0 ffff9a7884787570 233 lru 00000080:00200000:1.0:1787827240.457780:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000755b3d47 233: 1< 0 - - > 2< 954368 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457781:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000001662884b 00000080:00200000:1.0:1787827240.457782:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 234. 00000080:00200000:1.0:1787827240.457783:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 234 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457795:0:48374:0:(rw26.c:875:ll_write_end()) pos 958464, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457798:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002254d0bb[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 234] 00000080:00200000:1.0:1787827240.457799:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000ff863127 17ffffc0001021 2:0 ffff9a7884787828 234 lru 00000080:00200000:1.0:1787827240.457803:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000002db56ae6 234: 1< 0 - - > 2< 958464 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457804:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002254d0bb 00000080:00200000:1.0:1787827240.457805:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 235. 00000080:00200000:1.0:1787827240.457806:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 235 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457814:0:48374:0:(rw26.c:875:ll_write_end()) pos 962560, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457817:0:48374:0:(rw26.c:933:ll_write_end()) page@000000002a4a9973[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 235] 00000080:00200000:1.0:1787827240.457819:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000062d0b05e 17ffffc0001001 3:0 ffff9a7884787000 235 lru 00000080:00200000:1.0:1787827240.457823:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000001fea42a 235: 1< 0 - - > 2< 962560 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457824:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000002a4a9973 00000080:00200000:1.0:1787827240.457825:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 236. 00000080:00200000:1.0:1787827240.457826:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 236 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457850:0:48374:0:(rw26.c:875:ll_write_end()) pos 966656, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457855:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000cd7e2091[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 236] 00000080:00200000:1.0:1787827240.457858:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000fb944073 17ffffc0001001 3:0 ffff9a7884787cb0 236 lru 00000080:00200000:1.0:1787827240.457864:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000e51f993a 236: 1< 0 - - > 2< 966656 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457867:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000cd7e2091 00000080:00200000:1.0:1787827240.457868:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 237. 00000080:00200000:1.0:1787827240.457870:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 237 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457887:0:48374:0:(rw26.c:875:ll_write_end()) pos 970752, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457893:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000690c8b60[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 237] 00000080:00200000:1.0:1787827240.457897:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e7d88def 17ffffc0001001 3:0 ffff9a7884787ae0 237 lru 00000080:00200000:1.0:1787827240.457903:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000062f8e293 237: 1< 0 - - > 2< 970752 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457906:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000690c8b60 00000080:00200000:1.0:1787827240.457907:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 238. 00000080:00200000:1.0:1787827240.457909:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 238 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457925:0:48374:0:(rw26.c:875:ll_write_end()) pos 974848, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457928:0:48374:0:(rw26.c:933:ll_write_end()) page@000000000ab2c2f1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 238] 00000080:00200000:1.0:1787827240.457930:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000000c2bc1d3 17ffffc0001001 3:0 ffff9a7884787658 238 lru 00000080:00200000:1.0:1787827240.457934:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000473e0759 238: 1< 0 - - > 2< 974848 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457936:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000000ab2c2f1 00000080:00200000:1.0:1787827240.457937:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 239. 00000080:00200000:1.0:1787827240.457938:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 239 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457947:0:48374:0:(rw26.c:875:ll_write_end()) pos 978944, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457950:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000dacd34d1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 239] 00000080:00200000:1.0:1787827240.457952:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000eb0f74f6 17ffffc0001001 3:0 ffff9a78847879f8 239 lru 00000080:00200000:1.0:1787827240.457956:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000003ca8458b 239: 1< 0 - - > 2< 978944 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457958:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000dacd34d1 00000080:00200000:1.0:1787827240.457958:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 240. 00000080:00200000:1.0:1787827240.457959:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 240 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457968:0:48374:0:(rw26.c:875:ll_write_end()) pos 983040, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457970:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000046db5e79[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 240] 00000080:00200000:1.0:1787827240.457972:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000e113befc 17ffffc0001001 3:0 ffff9a7884787910 240 lru 00000080:00200000:1.0:1787827240.457975:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000071e7dea9 240: 1< 0 - - > 2< 983040 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.457977:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000046db5e79 00000080:00200000:1.0:1787827240.457978:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 241. 00000080:00200000:1.0:1787827240.457979:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 241 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.457989:0:48374:0:(rw26.c:875:ll_write_end()) pos 987136, len 4096, copied 4096 00000080:00200000:1.0:1787827240.457992:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000dd761230[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 241] 00000080:00200000:1.0:1787827240.457994:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000dfe6a80d 17ffffc0001001 3:0 ffff9a7884787d98 241 lru 00000080:00200000:1.0:1787827240.457998:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@0000000084f318ea 241: 1< 0 - - > 2< 987136 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458000:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000dd761230 00000080:00200000:1.0:1787827240.458000:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 242. 00000080:00200000:1.0:1787827240.458001:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 242 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458010:0:48374:0:(rw26.c:875:ll_write_end()) pos 991232, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458012:0:48374:0:(rw26.c:933:ll_write_end()) page@000000006aff32c5[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 242] 00000080:00200000:1.0:1787827240.458014:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000075136b71 17ffffc0001001 3:0 ffff9a78847873a0 242 lru 00000080:00200000:1.0:1787827240.458017:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000aa7e87f 242: 1< 0 - - > 2< 991232 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458019:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000006aff32c5 00000080:00200000:1.0:1787827240.458019:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 243. 00000080:00200000:1.0:1787827240.458020:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 243 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458035:0:48374:0:(rw26.c:875:ll_write_end()) pos 995328, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458038:0:48374:0:(rw26.c:933:ll_write_end()) page@000000003631f3fd[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 243] 00000080:00200000:1.0:1787827240.458040:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a90f31c9 17ffffc0001001 3:0 ffff9a7889102740 243 lru 00000080:00200000:1.0:1787827240.458043:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000d1ae1068 243: 1< 0 - - > 2< 995328 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458045:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000003631f3fd 00000080:00200000:1.0:1787827240.458046:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 244. 00000080:00200000:1.0:1787827240.458047:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 244 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458055:0:48374:0:(rw26.c:875:ll_write_end()) pos 999424, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458058:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d74c2417[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 244] 00000080:00200000:1.0:1787827240.458059:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000007aa6e3b5 17ffffc0001001 3:0 ffff9a7889102e80 244 lru 00000080:00200000:1.0:1787827240.458063:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000915ddc37 244: 1< 0 - - > 2< 999424 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458064:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d74c2417 00000080:00200000:1.0:1787827240.458065:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 245. 00000080:00200000:1.0:1787827240.458067:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 245 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458075:0:48374:0:(rw26.c:875:ll_write_end()) pos 1003520, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458078:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000fbb38322[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 245] 00000080:00200000:1.0:1787827240.458080:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000004c163c49 17ffffc0001001 3:0 ffff9a7889102488 245 lru 00000080:00200000:1.0:1787827240.458084:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000c9eb1ca1 245: 1< 0 - - > 2< 1003520 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458086:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000fbb38322 00000080:00200000:1.0:1787827240.458087:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 246. 00000080:00200000:1.0:1787827240.458088:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 246 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458097:0:48374:0:(rw26.c:875:ll_write_end()) pos 1007616, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458100:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000cd5d666b[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 246] 00000080:00200000:1.0:1787827240.458102:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000072dbe59e 17ffffc0001001 3:0 ffff9a7889102bc8 246 lru 00000080:00200000:1.0:1787827240.458106:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000d9d2b9e6 246: 1< 0 - - > 2< 1007616 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458107:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000cd5d666b 00000080:00200000:1.0:1787827240.458108:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 247. 00000080:00200000:1.0:1787827240.458109:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 247 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458117:0:48374:0:(rw26.c:875:ll_write_end()) pos 1011712, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458120:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000cbcaeb26[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 247] 00000080:00200000:1.0:1787827240.458121:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000a56d328b 17ffffc0001001 3:0 ffff9a7889102570 247 lru 00000080:00200000:1.0:1787827240.458125:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000bec90071 247: 1< 0 - - > 2< 1011712 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458126:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000cbcaeb26 00000080:00200000:1.0:1787827240.458127:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 248. 00000080:00200000:1.0:1787827240.458128:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 248 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458137:0:48374:0:(rw26.c:875:ll_write_end()) pos 1015808, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458140:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000517014ba[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 248] 00000080:00200000:1.0:1787827240.458142:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @000000001fa3a3fc 17ffffc0001001 3:0 ffff9a7889102828 248 lru 00000080:00200000:1.0:1787827240.458146:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000000cd6a6ec 248: 1< 0 - - > 2< 1015808 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458147:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000517014ba 00000080:00200000:1.0:1787827240.458148:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 249. 00000080:00200000:1.0:1787827240.458149:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 249 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458161:0:48374:0:(rw26.c:875:ll_write_end()) pos 1019904, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458164:0:48374:0:(rw26.c:933:ll_write_end()) page@00000000d5b83658[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 249] 00000080:00200000:1.0:1787827240.458166:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000fbf94313 17ffffc0001021 2:0 ffff9a7889102000 249 lru 00000080:00200000:1.0:1787827240.458169:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000443e5cbb 249: 1< 0 - - > 2< 1019904 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458171:0:48374:0:(rw26.c:933:ll_write_end()) end page@00000000d5b83658 00000080:00200000:1.0:1787827240.458172:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 250. 00000080:00200000:1.0:1787827240.458173:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 250 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458182:0:48374:0:(rw26.c:875:ll_write_end()) pos 1024000, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458185:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000013419bfc[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 250] 00000080:00200000:1.0:1787827240.458187:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000405f1960 17ffffc0001001 3:0 ffff9a7889102cb0 250 lru 00000080:00200000:1.0:1787827240.458190:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000296cfd17 250: 1< 0 - - > 2< 1024000 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458192:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000013419bfc 00000080:00200000:1.0:1787827240.458193:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 251. 00000080:00200000:1.0:1787827240.458194:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 251 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458202:0:48374:0:(rw26.c:875:ll_write_end()) pos 1028096, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458204:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000022b431bf[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 251] 00000080:00200000:1.0:1787827240.458206:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000789596f2 17ffffc0001001 3:0 ffff9a7889102ae0 251 lru 00000080:00200000:1.0:1787827240.458209:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000005f61d384 251: 1< 0 - - > 2< 1028096 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458211:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000022b431bf 00000080:00200000:1.0:1787827240.458212:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 252. 00000080:00200000:1.0:1787827240.458213:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 252 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458222:0:48374:0:(rw26.c:875:ll_write_end()) pos 1032192, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458225:0:48374:0:(rw26.c:933:ll_write_end()) page@000000005f3143c1[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 252] 00000080:00200000:1.0:1787827240.458227:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000bf9fdf50 17ffffc0001001 3:0 ffff9a7889102658 252 lru 00000080:00200000:1.0:1787827240.458231:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000771169b3 252: 1< 0 - - > 2< 1032192 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458233:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000005f3143c1 00000080:00200000:1.0:1787827240.458233:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 253. 00000080:00200000:1.0:1787827240.458235:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 253 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458243:0:48374:0:(rw26.c:875:ll_write_end()) pos 1036288, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458245:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000046e57f36[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 253] 00000080:00200000:1.0:1787827240.458247:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @0000000033248ec1 17ffffc0001001 3:0 ffff9a78891029f8 253 lru 00000080:00200000:1.0:1787827240.458251:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000be4aa489 253: 1< 0 - - > 2< 1036288 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458253:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000046e57f36 00000080:00200000:1.0:1787827240.458253:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 254. 00000080:00200000:1.0:1787827240.458255:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 254 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458263:0:48374:0:(rw26.c:875:ll_write_end()) pos 1040384, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458266:0:48374:0:(rw26.c:933:ll_write_end()) page@000000000af8b432[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 254] 00000080:00200000:1.0:1787827240.458268:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000dfb9a941 17ffffc0001001 3:0 ffff9a7889102910 254 lru 00000080:00200000:1.0:1787827240.458272:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@000000005edd30b7 254: 1< 0 - - > 2< 1040384 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458273:0:48374:0:(rw26.c:933:ll_write_end()) end page@000000000af8b432 00000080:00200000:1.0:1787827240.458274:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 255. 00000080:00200000:1.0:1787827240.458275:0:48374:0:(rw26.c:681:ll_write_begin()) Writing 255 of 0 to 4096 bytes 00000080:00200000:1.0:1787827240.458284:0:48374:0:(rw26.c:875:ll_write_end()) pos 1044480, len 4096, copied 4096 00000080:00200000:1.0:1787827240.458287:0:48374:0:(rw26.c:933:ll_write_end()) page@0000000064038b0a[3 000000006a7cba22 2 1 00000000f1b7f9f4 0 0 255] 00000080:00200000:1.0:1787827240.458288:0:48374:0:(rw26.c:933:ll_write_end()) vmpage @00000000edbe670b 17ffffc0001001 3:0 ffff9a7889102d98 255 lru 00000080:00200000:1.0:1787827240.458292:0:48374:0:(rw26.c:933:ll_write_end()) osc-page@00000000b4e50311 255: 1< 0 - - > 2< 1044480 0 0 0x0 0x0 | 0000000023a82262 00000000fa054274 > 3< 0 0 > 4< 0 0 8 5951488 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:1.0:1787827240.458293:0:48374:0:(rw26.c:933:ll_write_end()) end page@0000000064038b0a 00000080:00200000:1.0:1787827240.458294:0:48374:0:(rw26.c:933:ll_write_end()) queued cl_page: 256. 00000080:00200000:1.0:1787827240.458297:0:48374:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 256, from 0, to 4096 prio 0 00000008:00010000:1.0:1787827240.458341:0:48374:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458361:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458368:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458374:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458379:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458384:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458394:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458399:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458404:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458409:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458414:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458419:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458424:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458429:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458434:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.458446:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.458451:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458457:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458462:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458467:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458472:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458477:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458482:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458487:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458496:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458501:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458506:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458528:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458533:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458539:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458544:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.458552:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.458555:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458564:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458570:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458575:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458579:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458584:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458589:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458594:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458599:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458604:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458610:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458616:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458621:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458626:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458631:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.458638:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.458641:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458646:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458651:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458659:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458665:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458670:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458675:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458680:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458684:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458689:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458694:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458699:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458706:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458711:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458716:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.458723:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.458726:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458731:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458736:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458741:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458746:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458754:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458759:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458764:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458769:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458774:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458778:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458783:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458788:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458798:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458808:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.458822:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.458827:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458858:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458868:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458876:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458883:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458892:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458900:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458914:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458922:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458932:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458942:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458951:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458961:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458971:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458977:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.458985:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.458989:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.458996:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459001:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459006:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459011:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459016:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459021:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459026:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459034:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459050:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459061:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459071:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459080:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459090:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459100:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.459114:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.459121:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459131:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459148:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459159:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459169:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459179:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459190:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459201:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459210:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459219:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459228:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459238:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459249:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459262:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459275:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.459293:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.459302:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459315:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459328:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459340:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459358:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459370:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459383:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459395:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459408:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459419:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459430:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459441:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459452:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459466:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459480:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.459497:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.459507:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459538:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459550:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459562:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459575:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459586:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459604:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459616:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459629:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459641:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459651:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459660:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459669:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459678:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459686:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.459700:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.459707:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459717:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459725:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459734:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459744:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459754:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459764:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459775:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459792:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459802:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459812:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459822:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459849:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459856:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459861:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.459870:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.459873:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459883:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459889:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459893:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459898:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459903:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459908:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459913:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459918:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459923:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459929:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459935:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459940:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459945:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459950:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.459957:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.459960:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459965:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459970:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459978:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459983:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459988:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459993:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.459998:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460003:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460007:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460012:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460017:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460023:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460027:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460032:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.460039:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.460042:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460047:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460052:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460057:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460061:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460070:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460075:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460079:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460084:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460089:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460094:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460099:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460104:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460109:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460115:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.460122:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.460125:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460130:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460135:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460140:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460145:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460150:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460155:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460164:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460169:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460174:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460179:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460184:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460189:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460193:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460198:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.460205:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.460209:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460214:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460219:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460223:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460228:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460233:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460238:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460243:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460248:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460256:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460261:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460266:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460271:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460276:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460280:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.460287:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.460290:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460295:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460309:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460319:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460329:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460337:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460345:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460353:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460362:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460371:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460379:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460389:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460397:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460405:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.460414:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.460425:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 15, dirtied 15 00000008:00010000:1.0:1787827240.460431:0:48374:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000dbe11f66 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1787827240.460440:0:48374:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a78916f92f8, count 1, dirtied 1 00000080:00200000:1.0:1787827240.460444:0:48374:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 256 pages 1048576 bytes, tot: 1048576 00000080:00200000:1.0:1787827240.460448:0:48374:0:(vvp_io.c:1421:vvp_io_write_start()) f2: write bytes 0, result: 1048576 00000020:00010000:1.0:1787827240.460468:0:48374:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_release():245 00000020:00010000:1.0:1787827240.460472:0:48374:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1787827240.460476:0:48374:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7886af0bb0 (ffff9a7885afd488/1) at cl_lock_cancel():142 00010000:00010000:1.0:1787827240.460485:0:48374:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.460499:0:48374:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000db3ea405/0xebdcd48a4298e488 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dabd0 expref: -99 pid: 48374 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:1.0:1787827240.460532:0:48374:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a788405fd58 (ffff9a7885afd7e8/0) at cl_lock_fini():72 00000020:00010000:1.0:1787827240.460536:0:48374:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7886af0bb0 (ffff9a7885afd7e8/1) at cl_lock_fini():72 00000080:00200000:1.0:1787827240.460546:0:48374:0:(vvp_io.c:533:vvp_io_advance()) advancing 1048576 bytes 00000080:00200000:1.0:1787827240.460554:0:48374:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 255] 00000080:00200000:1.0:1787827240.460564:0:48374:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x52:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787827240.460567:0:48374:0:(file.c:2240:ll_file_io_generic()) f2: 2 io complete with rc: 0, result: 1048576, restart: 0 00000080:00200000:1.0:1787827240.460580:0:48374:0:(file.c:1775:ll_merge_attr_nolock()) [0x200001b72:0x52:0x0] updating i_size 1048576 i_blocks 0 00000080:00200000:1.0:1787827240.460586:0:48374:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 1048576 00000080:00200008:1.0:1787827240.460617:0:48374:0:(file.c:479:ll_file_release()) START file f2:[0x200001b72:0x52:0x0](ffff9a78916f9148), flags 100001 00000002:00100000:1.0:1787827240.460687:0:48374:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a67ef840 x1874671476530816/t12884902249(12884902249) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787827240.460697:0:48374:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787827240.460702:0:48374:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ede00 x1874671476531200/t0(0) o35->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787827240.460722:0:48374:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ede00 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48374:1874671476531200:192.168.203.140@tcp:35:dd.0 00000100:00100000:1.0:1787827240.460801:0:48374:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.462090:0:48374:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ede00 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48374:1874671476531200:192.168.203.140@tcp:35:dd.0 00000002:00100000:1.0:1787827240.462109:0:48374:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a67ef840 x1874671476530816/t12884902249(12884902249) o101->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827256 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787827240.467585:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.467608:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.467627:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.467646:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.467652:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.467659:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.467662:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.467669:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.467672:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.467679:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.467691:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.467698:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.467702:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.467705:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.467708:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.467715:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub0, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.467718:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.467721:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.467725:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub0, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.467728:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub0, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.467781:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.467791:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a67ec3c0 x1874671476531456/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.467816:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67ec3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476531456:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.467910:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.469663:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67ec3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476531456:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.469696:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x53:0x0](ffff9a78916fd348) 00000080:00200000:1.0:1787827240.469729:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub0, unhashed 1 00000080:00200000:1.0:1787827240.469755:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.469764:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.469773:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.469781:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.469787:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.469790:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.469792:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.469796:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.469799:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.469804:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.469811:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.469814:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.469817:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.469822:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.469825:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.469858:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub1, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.469862:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.469867:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.469871:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub1, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.469874:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub1, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.469904:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.469911:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66ffc00 x1874671476531712/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.469927:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66ffc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476531712:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.469969:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.471043:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66ffc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476531712:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.471068:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x54:0x0](ffff9a78916ff448) 00000080:00200000:1.0:1787827240.471086:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub1, unhashed 1 00000080:00200000:1.0:1787827240.471109:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.471117:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.471123:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.471131:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.471144:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.471147:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.471149:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.471155:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.471157:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.471163:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.471170:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.471173:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.471175:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.471177:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.471178:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.471186:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub2, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.471188:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.471191:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.471193:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub2, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.471195:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub2, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.471231:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.471236:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3c000 x1874671476531968/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.471247:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476531968:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.471277:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.472334:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476531968:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.472350:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x55:0x0](ffff9a78916fc2c8) 00000080:00200000:1.0:1787827240.472363:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub2, unhashed 1 00000080:00200000:1.0:1787827240.472380:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.472387:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.472393:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.472401:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.472406:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.472409:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.472410:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.472414:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.472416:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.472422:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.472429:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.472433:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.472436:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.472439:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.472442:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.472452:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub3, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.472455:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.472460:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.472463:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub3, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.472467:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub3, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.472493:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.472498:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3c3c0 x1874671476532224/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.472526:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532224:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.472558:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.473688:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532224:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.473703:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x56:0x0](ffff9a78916fec08) 00000080:00200000:1.0:1787827240.473723:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub3, unhashed 1 00000080:00200000:1.0:1787827240.473740:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.473746:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.473752:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.473759:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.473764:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.473768:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.473769:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.473773:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.473775:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.473780:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.473786:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.473788:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.473791:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.473793:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.473795:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.473801:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub4, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.473803:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.473806:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.473808:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub4, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.473810:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub4, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.473827:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.473849:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3fc00 x1874671476532480/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.473865:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532480:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.473901:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.474868:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532480:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.474892:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x57:0x0](ffff9a78916fb248) 00000080:00200000:1.0:1787827240.474908:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub4, unhashed 1 00000080:00200000:1.0:1787827240.474927:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.474937:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.474946:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.474957:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.474965:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.474969:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.474971:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.474975:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.474978:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.474983:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.474990:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.474992:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.474995:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.474997:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.474998:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.475005:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub5, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.475007:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.475010:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.475013:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub5, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.475015:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub5, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.475042:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.475047:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3f480 x1874671476532736/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.475058:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532736:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.475098:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.476079:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532736:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.476096:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x58:0x0](ffff9a78916fa1c8) 00000080:00200000:1.0:1787827240.476108:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub5, unhashed 1 00000080:00200000:1.0:1787827240.476126:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.476134:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.476141:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.476149:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.476154:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.476157:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.476159:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.476162:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.476164:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.476170:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.476176:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.476179:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.476182:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.476184:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.476185:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.476192:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub6, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.476195:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.476197:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.476200:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub6, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.476202:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub6, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.476220:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.476226:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3de00 x1874671476532992/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.476236:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532992:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.476268:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.477247:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476532992:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.477265:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x59:0x0](ffff9a789dd921c8) 00000080:00200000:1.0:1787827240.477278:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub6, unhashed 1 00000080:00200000:1.0:1787827240.477294:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.477301:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.477308:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.477315:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.477320:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.477323:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.477325:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.477328:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.477330:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.477337:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.477343:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.477345:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.477348:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.477350:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.477352:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.477358:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub7, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.477360:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.477363:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.477365:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub7, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.477367:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub7, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.477384:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.477389:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3e580 x1874671476533248/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.477399:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476533248:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.477429:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.478370:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476533248:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.478385:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x5a:0x0](ffff9a789dd900c8) 00000080:00200000:1.0:1787827240.478397:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub7, unhashed 1 00000080:00200000:1.0:1787827240.478413:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.478419:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.478426:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.478433:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.478438:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.478440:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.478443:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.478446:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.478449:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.478454:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.478461:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.478463:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.478465:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.478467:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.478469:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.478475:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub8, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.478477:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.478480:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.478482:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub8, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.478484:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub8, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.478501:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.478506:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3cb40 x1874671476533504/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.478535:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3cb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476533504:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.478568:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.479594:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3cb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476533504:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.479615:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x5b:0x0](ffff9a789dd93248) 00000080:00200000:1.0:1787827240.479627:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub8, unhashed 1 00000080:00200000:1.0:1787827240.479644:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.479651:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.479657:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.479665:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.479670:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.479673:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.479675:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.479678:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.479680:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.479686:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.479692:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.479695:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.479697:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.479699:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.479701:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.479707:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub9, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.479717:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.479720:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.479723:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub9, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.479725:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub9, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.479743:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.479748:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3f840 x1874671476533760/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.479759:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476533760:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.479786:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.480937:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476533760:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.480959:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x5c:0x0](ffff9a789dd96c08) 00000080:00200000:1.0:1787827240.480973:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub9, unhashed 1 00000080:00200000:1.0:1787827240.480993:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.481002:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.481011:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.481022:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.481030:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.481035:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.481038:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.481043:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.481047:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.481056:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.481066:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.481070:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.481074:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.481077:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.481080:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.481088:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub10, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.481092:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.481096:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.481100:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub10, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.481103:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub10, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.481126:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.481132:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3e940 x1874671476534016/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.481148:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534016:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.481189:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.482128:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534016:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.482142:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x5d:0x0](ffff9a789dd942c8) 00000080:00200000:1.0:1787827240.482154:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub10, unhashed 1 00000080:00200000:1.0:1787827240.482170:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.482177:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.482183:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.482190:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.482195:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.482198:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.482200:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.482204:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.482206:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.482212:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.482218:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.482220:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.482223:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.482224:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.482226:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.482231:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub11, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.482233:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.482236:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.482238:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub11, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.482241:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub11, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.482257:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.482262:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3d680 x1874671476534272/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.482273:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3d680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534272:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.482302:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.483290:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3d680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534272:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.483306:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x5e:0x0](ffff9a789dd97448) 00000080:00200000:1.0:1787827240.483318:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub11, unhashed 1 00000080:00200000:1.0:1787827240.483334:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.483341:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.483347:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.483354:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.483359:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.483362:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.483364:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.483367:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.483369:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.483375:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.483381:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.483383:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.483391:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.483393:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.483395:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.483402:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub12, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.483404:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.483407:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.483409:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub12, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.483411:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub12, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.483429:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.483434:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3cf00 x1874671476534528/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.483444:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3cf00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534528:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.483471:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.484415:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3cf00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534528:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.484430:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x5f:0x0](ffff9a789dd95348) 00000080:00200000:1.0:1787827240.484443:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub12, unhashed 1 00000080:00200000:1.0:1787827240.484459:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.484465:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.484471:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.484478:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.484483:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.484486:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.484488:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.484491:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.484493:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.484500:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.484505:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.484522:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.484525:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.484528:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.484529:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.484537:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub13, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.484539:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.484542:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.484544:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub13, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.484547:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub13, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.484565:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.484570:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3ed00 x1874671476534784/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.484580:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3ed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534784:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.484622:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.485625:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3ed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476534784:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.485648:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x60:0x0](ffff9a789dd95b88) 00000080:00200000:1.0:1787827240.485664:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub13, unhashed 1 00000080:00200000:1.0:1787827240.485681:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.485688:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.485694:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.485701:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.485706:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.485716:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.485719:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.485722:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.485725:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.485730:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.485736:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.485739:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.485741:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.485743:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.485744:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.485750:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub14, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.485753:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.485756:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.485758:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub14, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.485760:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub14, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.485777:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.485782:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3f0c0 x1874671476535040/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.485793:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535040:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.485822:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.486725:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535040:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.486740:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x61:0x0](ffff9a789dd92a08) 00000080:00200000:1.0:1787827240.486755:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub14, unhashed 1 00000080:00200000:1.0:1787827240.486776:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.486785:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.486795:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.486805:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.486813:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.486818:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.486821:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.486826:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.486849:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.486859:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.486868:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.486871:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.486874:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.486876:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.486878:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.486886:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub15, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.486888:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.486891:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.486893:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub15, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.486896:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub15, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.486917:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.486924:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3e1c0 x1874671476535296/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.486939:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535296:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.486975:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.487965:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535296:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.487985:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x62:0x0](ffff9a789dd91988) 00000080:00200000:1.0:1787827240.488001:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub15, unhashed 1 00000080:00200000:1.0:1787827240.488017:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.488024:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.488030:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.488038:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.488043:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.488046:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.488047:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.488051:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.488053:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.488059:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.488065:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.488067:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.488070:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.488072:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.488074:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.488079:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub16, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.488081:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.488084:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.488086:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub16, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.488088:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub16, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.488106:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.488111:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3d2c0 x1874671476535552/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.488121:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535552:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.488150:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.489043:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535552:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.489059:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x63:0x0](ffff9a789dd90908) 00000080:00200000:1.0:1787827240.489083:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub16, unhashed 1 00000080:00200000:1.0:1787827240.489099:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.489105:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.489112:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.489119:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.489124:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.489126:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.489128:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.489132:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.489134:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.489140:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.489146:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.489149:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.489151:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.489152:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.489154:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.489160:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub17, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.489162:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.489165:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.489167:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub17, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.489169:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub17, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.489186:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.489190:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3c780 x1874671476535808/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.489200:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535808:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.489229:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.490415:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476535808:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.490431:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x64:0x0](ffff9a789dd963c8) 00000080:00200000:1.0:1787827240.490501:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub17, unhashed 1 00000080:00200000:1.0:1787827240.490534:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.490542:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.490549:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.490557:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.490562:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.490565:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.490567:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.490570:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.490573:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.490579:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.490585:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.490588:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.490590:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.490592:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.490594:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.490600:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub18, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.490602:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.490605:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.490607:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub18, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.490610:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub18, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.490627:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.490633:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3da40 x1874671476536064/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.490643:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536064:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.490671:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.491717:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536064:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.491740:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x65:0x0](ffff9a789dd94b08) 00000080:00200000:1.0:1787827240.491758:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub18, unhashed 1 00000080:00200000:1.0:1787827240.491784:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.491794:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.491801:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.491810:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.491816:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.491819:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.491821:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.491825:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.491827:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.491851:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.491858:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.491863:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.491867:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.491869:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.491873:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.491882:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub19, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.491884:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.491888:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.491890:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub19, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.491892:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub19, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.491925:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.491930:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6533c00 x1874671476536320/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.491941:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536320:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.491974:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.492986:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536320:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.493002:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x66:0x0](ffff9a789dd93a88) 00000080:00200000:1.0:1787827240.493015:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub19, unhashed 1 00000080:00200000:1.0:1787827240.493033:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.493039:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.493046:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.493061:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.493067:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.493070:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.493072:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.493076:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.493078:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.493084:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.493090:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.493093:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.493095:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.493097:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.493098:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.493104:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub20, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.493106:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.493109:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.493112:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub20, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.493114:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub20, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.493132:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.493137:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6533480 x1874671476536576/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.493147:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536576:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.493177:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.494067:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536576:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.494083:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x67:0x0](ffff9a789dd91148) 00000080:00200000:1.0:1787827240.494095:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub20, unhashed 1 00000080:00200000:1.0:1787827240.494111:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.494118:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.494124:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.494131:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.494136:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.494139:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.494141:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.494144:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.494146:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.494152:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.494157:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.494160:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.494162:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.494164:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.494166:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.494172:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub21, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.494174:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.494177:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.494179:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub21, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.494181:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub21, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.494198:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.494203:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6531e00 x1874671476536832/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.494213:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536832:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.494257:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.495185:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476536832:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.495216:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x68:0x0](ffff9a789defdb88) 00000080:00200000:1.0:1787827240.495229:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub21, unhashed 1 00000080:00200000:1.0:1787827240.495248:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.495256:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.495263:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.495272:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.495277:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.495280:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.495282:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.495286:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.495288:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.495295:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.495301:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.495304:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.495308:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.495310:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.495311:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.495319:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub22, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.495321:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.495324:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.495327:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub22, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.495329:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub22, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.495348:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.495353:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6532580 x1874671476537088/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.495365:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537088:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.495398:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.496356:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537088:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.496377:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x69:0x0](ffff9a789defaa08) 00000080:00200000:1.0:1787827240.496393:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub22, unhashed 1 00000080:00200000:1.0:1787827240.496415:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.496424:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.496434:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.496446:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.496455:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.496459:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.496462:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.496468:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.496472:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.496481:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.496491:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.496495:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.496499:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.496502:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.496505:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.496528:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub23, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.496532:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.496537:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.496541:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub23, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.496545:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub23, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.496569:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.496576:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6530b40 x1874671476537344/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.496591:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537344:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.496626:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.497567:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537344:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.497592:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x6a:0x0](ffff9a789def9988) 00000080:00200000:1.0:1787827240.497610:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub23, unhashed 1 00000080:00200000:1.0:1787827240.497630:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.497637:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.497645:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.497653:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.497659:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.497662:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.497664:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.497668:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.497670:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.497676:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.497682:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.497685:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.497688:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.497689:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.497691:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.497697:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub24, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.497699:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.497702:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.497704:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub24, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.497707:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub24, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.497726:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.497733:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6533840 x1874671476537600/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.497749:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537600:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.497791:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.498846:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537600:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.498863:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x6b:0x0](ffff9a789def8908) 00000080:00200000:1.0:1787827240.498876:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub24, unhashed 1 00000080:00200000:1.0:1787827240.498893:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.498900:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.498906:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.498914:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.498919:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.498922:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.498924:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.498927:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.498929:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.498935:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.498942:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.498944:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.498947:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.498949:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.498951:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.498956:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub25, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.498959:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.498961:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.498964:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub25, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.498966:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub25, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.498985:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.498990:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6532940 x1874671476537856/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.499002:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537856:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.499034:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.499984:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476537856:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.500001:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x6c:0x0](ffff9a789defe3c8) 00000080:00200000:1.0:1787827240.500013:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub25, unhashed 1 00000080:00200000:1.0:1787827240.500030:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.500036:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.500042:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.500049:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.500054:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.500057:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.500059:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.500063:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.500065:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.500071:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.500077:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.500080:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.500082:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.500083:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.500085:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.500091:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub26, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.500093:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.500095:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.500098:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub26, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.500100:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub26, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.500117:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.500122:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6531680 x1874671476538112/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.500133:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538112:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.500163:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.501196:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538112:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.501211:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x6d:0x0](ffff9a789defcb08) 00000080:00200000:1.0:1787827240.501222:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub26, unhashed 1 00000080:00200000:1.0:1787827240.501239:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.501246:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.501261:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.501269:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.501274:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.501277:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.501279:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.501282:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.501284:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.501290:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.501296:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.501299:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.501301:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.501303:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.501304:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.501310:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub27, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.501312:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.501315:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.501317:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub27, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.501320:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub27, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.501337:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.501342:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6530f00 x1874671476538368/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.501353:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538368:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.501386:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.502385:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538368:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.502401:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x6e:0x0](ffff9a789defba88) 00000080:00200000:1.0:1787827240.502412:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub27, unhashed 1 00000080:00200000:1.0:1787827240.502428:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.502434:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.502440:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.502447:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.502453:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.502455:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.502457:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.502460:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.502463:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.502468:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.502474:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.502477:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.502479:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.502481:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.502483:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.502490:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub28, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.502492:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.502494:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.502497:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub28, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.502499:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub28, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.502530:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.502535:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6532d00 x1874671476538624/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.502546:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538624:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.502579:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.503505:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538624:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.503541:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x6f:0x0](ffff9a789def9148) 00000080:00200000:1.0:1787827240.503574:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub28, unhashed 1 00000080:00200000:1.0:1787827240.503597:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.503606:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.503615:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.503625:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.503630:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.503634:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.503636:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.503640:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.503642:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.503648:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.503654:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.503656:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.503659:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.503661:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.503662:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.503670:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub29, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.503673:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.503675:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.503678:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub29, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.503680:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub29, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.503698:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.503703:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65330c0 x1874671476538880/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.503713:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65330c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538880:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.503760:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.504654:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65330c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476538880:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.504669:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x70:0x0](ffff9a789def80c8) 00000080:00200000:1.0:1787827240.504682:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub29, unhashed 1 00000080:00200000:1.0:1787827240.504698:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.504705:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.504711:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.504718:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.504724:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.504727:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.504730:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.504734:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.504738:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.504747:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.504757:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.504761:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.504765:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.504769:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.504771:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.504780:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub30, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.504784:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.504788:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.504791:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub30, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.504793:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub30, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.504813:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.504818:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65321c0 x1874671476539136/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.504873:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539136:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.504903:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.505871:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539136:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.505887:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x71:0x0](ffff9a789defa1c8) 00000080:00200000:1.0:1787827240.505899:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub30, unhashed 1 00000080:00200000:1.0:1787827240.505916:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.505922:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.505929:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.505936:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.505941:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.505944:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.505946:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.505950:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.505953:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.505962:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.505972:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.505977:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.505981:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.505984:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.505987:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.505997:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub31, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.506000:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.506005:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.506008:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub31, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.506010:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub31, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.506030:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.506035:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65312c0 x1874671476539392/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.506045:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65312c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539392:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.506081:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.507043:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65312c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539392:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.507058:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x72:0x0](ffff9a789defb248) 00000080:00200000:1.0:1787827240.507070:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub31, unhashed 1 00000080:00200000:1.0:1787827240.507086:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.507092:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.507099:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.507106:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.507111:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.507114:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.507116:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.507119:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.507122:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.507127:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.507134:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.507136:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.507138:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.507140:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.507142:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.507148:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub32, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.507150:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.507153:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.507155:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub32, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.507157:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub32, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.507174:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.507179:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6530780 x1874671476539648/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.507189:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539648:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.507219:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.508339:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539648:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.508357:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x73:0x0](ffff9a789defec08) 00000080:00200000:1.0:1787827240.508372:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub32, unhashed 1 00000080:00200000:1.0:1787827240.508395:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.508405:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.508414:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.508422:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.508427:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.508431:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.508433:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.508436:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.508438:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.508444:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.508450:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.508453:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.508455:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.508457:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.508459:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.508466:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub33, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.508468:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.508471:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.508473:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub33, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.508475:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub33, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.508493:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.508498:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6531a40 x1874671476539904/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.508525:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539904:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.508555:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.509681:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476539904:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.509699:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x74:0x0](ffff9a789defc2c8) 00000080:00200000:1.0:1787827240.509720:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub33, unhashed 1 00000080:00200000:1.0:1787827240.509741:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.509752:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.509763:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.509785:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.509795:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.509800:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.509804:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.509810:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.509814:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.509825:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.509860:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.509866:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.509871:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.509875:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.509878:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.509889:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub34, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.509893:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.509898:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.509902:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub34, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.509905:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub34, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.509935:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.509942:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6530000 x1874671476540160/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.509960:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540160:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.510000:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.511131:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540160:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.511153:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x75:0x0](ffff9a789deff448) 00000080:00200000:1.0:1787827240.511169:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub34, unhashed 1 00000080:00200000:1.0:1787827240.511199:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.511208:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.511217:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.511230:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.511239:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.511243:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.511246:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.511251:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.511256:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.511265:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.511275:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.511279:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.511283:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.511286:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.511288:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.511303:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub35, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.511307:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.511311:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.511315:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub35, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.511319:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub35, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.511342:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.511350:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65303c0 x1874671476540416/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.511365:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540416:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.511400:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.512463:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540416:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.512484:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x76:0x0](ffff9a789defd348) 00000080:00200000:1.0:1787827240.512499:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub35, unhashed 1 00000080:00200000:1.0:1787827240.512535:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.512545:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.512554:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.512566:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.512575:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.512579:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.512583:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.512587:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.512591:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.512601:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.512611:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.512615:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.512618:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.512621:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.512624:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.512633:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub36, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.512636:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.512641:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.512645:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub36, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.512648:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub36, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.512690:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.512698:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667da40 x1874671476540672/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.512714:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667da40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540672:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.512751:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.513682:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667da40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540672:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.513710:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x77:0x0](ffff9a789dfef448) 00000080:00200000:1.0:1787827240.513731:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub36, unhashed 1 00000080:00200000:1.0:1787827240.513748:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.513754:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.513763:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.513782:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.513791:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.513795:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.513798:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.513803:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.513807:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.513816:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.513824:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.513827:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.513848:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.513850:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.513852:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.513859:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub37, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.513861:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.513864:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.513866:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub37, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.513868:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub37, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.513888:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.513893:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667c000 x1874671476540928/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.513904:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667c000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540928:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.513943:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.514993:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667c000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476540928:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.515009:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x78:0x0](ffff9a789dfed348) 00000080:00200000:1.0:1787827240.515020:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub37, unhashed 1 00000080:00200000:1.0:1787827240.515045:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.515051:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.515058:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.515069:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.515074:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.515077:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.515079:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.515083:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.515085:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.515091:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.515097:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.515099:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.515102:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.515103:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.515105:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.515110:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub38, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.515113:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.515115:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.515118:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub38, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.515120:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub38, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.515137:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.515142:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667c3c0 x1874671476541184/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.515153:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541184:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.515180:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.516286:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541184:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.516309:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x79:0x0](ffff9a789dfedb88) 00000080:00200000:1.0:1787827240.516326:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub38, unhashed 1 00000080:00200000:1.0:1787827240.516350:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.516361:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.516371:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.516382:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.516390:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.516395:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.516398:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.516403:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.516406:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.516416:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.516426:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.516431:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.516435:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.516438:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.516441:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.516451:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub39, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.516454:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.516459:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.516463:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub39, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.516466:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub39, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.516494:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.516502:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667fc00 x1874671476541440/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.516537:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667fc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541440:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.516576:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.517530:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667fc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541440:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.517554:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x7a:0x0](ffff9a789dfeaa08) 00000080:00200000:1.0:1787827240.517573:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub39, unhashed 1 00000080:00200000:1.0:1787827240.517596:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.517606:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.517616:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.517628:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.517636:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.517640:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.517644:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.517648:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.517652:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.517661:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.517672:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.517677:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.517682:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.517685:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.517688:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.517697:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub40, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.517701:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.517706:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.517709:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub40, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.517713:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub40, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.517739:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.517746:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667f480 x1874671476541696/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.517763:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667f480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541696:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.517801:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.518898:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667f480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541696:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.518930:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x7b:0x0](ffff9a789dfe9988) 00000080:00200000:1.0:1787827240.518947:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub40, unhashed 1 00000080:00200000:1.0:1787827240.518970:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.518979:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.518989:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.519000:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.519008:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.519013:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.519016:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.519021:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.519024:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.519043:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.519054:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.519059:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.519063:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.519067:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.519070:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.519078:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub41, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.519082:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.519086:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.519090:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub41, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.519093:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub41, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.519117:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.519124:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667de00 x1874671476541952/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.519140:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667de00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541952:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.519178:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.520116:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667de00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476541952:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.520140:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x7c:0x0](ffff9a789dfe8908) 00000080:00200000:1.0:1787827240.520159:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub41, unhashed 1 00000080:00200000:1.0:1787827240.520182:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.520193:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.520202:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.520213:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.520221:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.520226:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.520229:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.520234:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.520238:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.520247:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.520257:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.520263:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.520267:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.520271:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.520274:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.520283:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub42, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.520287:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.520291:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.520295:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub42, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.520298:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub42, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.520325:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.520333:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667e580 x1874671476542208/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.520349:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667e580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542208:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.520390:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.521404:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667e580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542208:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.521420:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x7d:0x0](ffff9a789dfee3c8) 00000080:00200000:1.0:1787827240.521433:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub42, unhashed 1 00000080:00200000:1.0:1787827240.521450:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.521457:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.521464:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.521473:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.521479:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.521482:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.521484:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.521488:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.521490:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.521496:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.521502:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.521505:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.521522:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.521525:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.521527:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.521535:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub43, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.521538:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.521541:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.521543:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub43, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.521545:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub43, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.521565:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.521570:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667cb40 x1874671476542464/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.521581:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667cb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542464:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.521611:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.522625:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667cb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542464:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.522640:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x7e:0x0](ffff9a789dfecb08) 00000080:00200000:1.0:1787827240.522652:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub43, unhashed 1 00000080:00200000:1.0:1787827240.522668:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.522675:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.522681:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.522689:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.522694:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.522697:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.522700:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.522703:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.522705:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.522720:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.522727:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.522730:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.522732:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.522734:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.522736:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.522742:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub44, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.522744:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.522747:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.522750:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub44, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.522752:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub44, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.522770:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.522775:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667f840 x1874671476542720/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.522785:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667f840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542720:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.522815:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.523822:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667f840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542720:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.523856:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x7f:0x0](ffff9a789dfeba88) 00000080:00200000:1.0:1787827240.523884:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub44, unhashed 1 00000080:00200000:1.0:1787827240.523908:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.523915:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.523922:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.523929:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.523935:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.523938:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.523940:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.523943:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.523946:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.523953:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.523963:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.523967:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.523971:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.523974:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.523977:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.523986:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub45, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.523989:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.523994:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.523998:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub45, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.524001:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub45, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.524024:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.524037:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667e940 x1874671476542976/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.524048:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667e940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542976:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.524087:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.525230:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667e940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476542976:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.525246:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x80:0x0](ffff9a789dfe9148) 00000080:00200000:1.0:1787827240.525259:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub45, unhashed 1 00000080:00200000:1.0:1787827240.525276:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.525283:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.525290:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.525299:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.525304:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.525307:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.525309:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.525313:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.525316:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.525322:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.525328:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.525331:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.525334:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.525336:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.525338:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.525344:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub46, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.525346:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.525349:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.525351:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub46, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.525354:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub46, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.525371:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.525376:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667d680 x1874671476543232/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.525387:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667d680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476543232:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.525418:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.526537:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667d680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476543232:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.526563:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x81:0x0](ffff9a789dfe80c8) 00000080:00200000:1.0:1787827240.526581:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub46, unhashed 1 00000080:00200000:1.0:1787827240.526605:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.526615:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.526624:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.526636:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.526644:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.526649:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.526652:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.526658:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.526662:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.526671:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.526682:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.526688:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.526692:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.526696:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.526699:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.526708:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub47, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.526723:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.526729:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.526733:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub47, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.526736:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub47, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.526765:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.526772:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667cf00 x1874671476543488/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.526787:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667cf00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476543488:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.526824:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.527888:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667cf00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476543488:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.527919:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x82:0x0](ffff9a789dfea1c8) 00000080:00200000:1.0:1787827240.527938:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub47, unhashed 1 00000080:00200000:1.0:1787827240.527962:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.527972:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.527981:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.527993:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.528001:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.528006:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.528009:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.528013:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.528017:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.528027:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.528045:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.528050:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.528054:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.528057:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.528060:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.528069:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub48, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.528073:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.528078:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.528082:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub48, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.528085:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub48, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.528112:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.528120:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667ed00 x1874671476543744/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.528137:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667ed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476543744:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.528176:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.529187:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667ed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476543744:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.529211:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x83:0x0](ffff9a789dfeb248) 00000080:00200000:1.0:1787827240.529228:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub48, unhashed 1 00000080:00200000:1.0:1787827240.529250:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.529259:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.529268:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.529279:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.529288:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.529292:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.529296:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.529301:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.529304:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.529323:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.529338:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.529343:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.529348:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.529351:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.529355:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.529363:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub49, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.529368:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.529372:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.529376:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub49, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.529379:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub49, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.529407:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.529427:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667f0c0 x1874671476544000/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.529445:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544000:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.529483:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.530632:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544000:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.530651:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x84:0x0](ffff9a789dfeec08) 00000080:00200000:1.0:1787827240.530663:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub49, unhashed 1 00000080:00200000:1.0:1787827240.530679:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.530685:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.530692:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.530699:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.530704:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.530707:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.530709:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.530713:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.530715:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.530721:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.530728:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.530730:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.530732:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.530734:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.530736:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.530743:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub50, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.530745:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.530748:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.530750:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub50, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.530752:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub50, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.530770:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.530775:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667e1c0 x1874671476544256/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.530785:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544256:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.530815:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.531801:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544256:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.531825:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x85:0x0](ffff9a789dfec2c8) 00000080:00200000:1.0:1787827240.531855:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub50, unhashed 1 00000080:00200000:1.0:1787827240.531873:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.531881:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.531888:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.531897:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.531910:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.531914:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.531916:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.531919:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.531922:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.531928:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.531934:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.531937:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.531939:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.531941:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.531943:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.531949:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub51, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.531951:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.531954:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.531957:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub51, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.531959:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub51, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.531977:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.531982:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667d2c0 x1874671476544512/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.531993:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544512:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.532021:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.532981:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544512:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.533015:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x86:0x0](ffff9a789ddd5348) 00000080:00200000:1.0:1787827240.533029:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub51, unhashed 1 00000080:00200000:1.0:1787827240.533045:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.533052:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.533058:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.533065:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.533071:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.533074:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.533076:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.533079:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.533081:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.533087:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.533094:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.533096:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.533099:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.533101:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.533102:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.533108:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub52, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.533110:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.533113:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.533115:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub52, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.533118:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub52, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.533135:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.533141:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667c780 x1874671476544768/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.533156:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667c780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544768:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.533203:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.534270:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667c780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476544768:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.534285:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x87:0x0](ffff9a789ddd5b88) 00000080:00200000:1.0:1787827240.534297:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub52, unhashed 1 00000080:00200000:1.0:1787827240.534313:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.534319:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.534325:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.534339:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.534345:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.534348:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.534350:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.534354:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.534356:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.534362:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.534368:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.534377:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.534380:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.534384:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.534386:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.534392:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub53, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.534394:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.534397:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.534399:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub53, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.534402:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub53, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.534436:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.534444:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a21c0 x1874671476545024/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.534459:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a21c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545024:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.534500:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.535557:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a21c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545024:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.535590:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x88:0x0](ffff9a789ddd2a08) 00000080:00200000:1.0:1787827240.535613:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub53, unhashed 1 00000080:00200000:1.0:1787827240.535644:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.535656:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.535670:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.535683:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.535693:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.535698:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.535702:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.535707:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.535711:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.535721:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.535733:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.535739:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.535744:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.535747:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.535751:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.535761:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub54, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.535766:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.535770:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.535774:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub54, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.535778:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub54, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.535822:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.535856:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a12c0 x1874671476545280/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.535876:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a12c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545280:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.535941:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.539078:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a12c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545280:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.539121:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x89:0x0](ffff9a789ddd1988) 00000080:00200000:1.0:1787827240.539156:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub54, unhashed 1 00000080:00200000:1.0:1787827240.539220:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.539242:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.539262:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.539280:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.539288:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.539306:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.539310:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.539318:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.539323:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.539334:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.539351:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.539359:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.539365:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.539368:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.539372:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.539383:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub55, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.539387:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.539392:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.539397:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub55, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.539401:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub55, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.539458:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.539465:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a0780 x1874671476545536/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.539484:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a0780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545536:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.539578:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.541074:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545536:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.541119:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x8a:0x0](ffff9a789ddd0908) 00000080:00200000:1.0:1787827240.541148:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub55, unhashed 1 00000080:00200000:1.0:1787827240.541199:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.541214:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.541228:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.541239:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.541245:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.541249:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.541252:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.541258:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.541261:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.541268:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.541283:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.541290:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.541295:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.541299:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.541302:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.541314:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub56, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.541317:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.541321:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.541324:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub56, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.541326:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub56, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.541370:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.541379:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a1a40 x1874671476545792/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.541402:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a1a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545792:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.541458:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.542716:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a1a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476545792:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.542738:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x8b:0x0](ffff9a789ddd63c8) 00000080:00200000:1.0:1787827240.542778:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub56, unhashed 1 00000080:00200000:1.0:1787827240.542805:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.542816:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.542826:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.542888:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.542896:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.542901:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.542904:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.542908:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.542911:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.542920:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.542930:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.542935:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.542939:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.542942:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.542945:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.542955:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub57, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.542958:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.542963:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.542967:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub57, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.542970:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub57, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.542995:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.543003:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a0000 x1874671476546048/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.543028:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a0000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546048:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.543067:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.544179:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546048:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.544203:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x8c:0x0](ffff9a789ddd4b08) 00000080:00200000:1.0:1787827240.544221:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub57, unhashed 1 00000080:00200000:1.0:1787827240.544246:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.544255:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.544264:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.544274:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.544282:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.544287:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.544290:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.544295:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.544298:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.544307:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.544318:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.544323:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.544327:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.544330:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.544333:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.544342:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub58, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.544346:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.544350:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.544354:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub58, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.544357:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub58, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.544382:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.544389:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a03c0 x1874671476546304/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.544405:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a03c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546304:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.544445:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.545523:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a03c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546304:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.545546:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x8d:0x0](ffff9a789ddd3a88) 00000080:00200000:1.0:1787827240.545564:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub58, unhashed 1 00000080:00200000:1.0:1787827240.545588:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.545598:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.545607:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.545618:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.545626:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.545630:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.545633:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.545638:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.545642:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.545650:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.545660:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.545664:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.545668:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.545671:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.545674:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.545682:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub59, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.545686:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.545690:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.545694:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub59, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.545697:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub59, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.545721:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.545728:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a3c00 x1874671476546560/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.545743:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a3c00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546560:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.545782:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.546781:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a3c00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546560:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.546798:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x8e:0x0](ffff9a789ddd1148) 00000080:00200000:1.0:1787827240.546811:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub59, unhashed 1 00000080:00200000:1.0:1787827240.546845:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.546853:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.546860:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.546868:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.546873:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.546877:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.546878:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.546882:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.546884:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.546890:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.546896:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.546899:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.546901:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.546903:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.546905:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.546911:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub60, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.546914:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.546918:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.546922:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub60, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.546925:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub60, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.546949:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.546969:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a3480 x1874671476546816/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.546984:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a3480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546816:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.547017:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.548012:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a3480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476546816:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.548030:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x8f:0x0](ffff9a789ddd00c8) 00000080:00200000:1.0:1787827240.548056:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub60, unhashed 1 00000080:00200000:1.0:1787827240.548076:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.548085:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.548094:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.548106:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.548114:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.548119:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.548122:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.548128:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.548131:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.548139:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.548146:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.548150:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.548153:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.548156:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.548159:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.548168:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub61, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.548171:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.548176:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.548179:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub61, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.548183:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub61, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.548209:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.548216:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a1e00 x1874671476547072/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.548232:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a1e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547072:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.548276:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.549348:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a1e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547072:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.549368:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x90:0x0](ffff9a789ddd21c8) 00000080:00200000:1.0:1787827240.549385:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub61, unhashed 1 00000080:00200000:1.0:1787827240.549408:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.549417:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.549426:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.549436:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.549443:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.549448:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.549451:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.549456:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.549460:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.549468:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.549478:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.549482:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.549486:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.549489:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.549491:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.549500:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub62, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.549503:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.549521:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.549525:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub62, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.549529:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub62, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.549555:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.549562:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a2580 x1874671476547328/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.549576:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a2580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547328:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.549615:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.550658:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a2580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547328:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.550678:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x91:0x0](ffff9a789ddd3248) 00000080:00200000:1.0:1787827240.550694:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub62, unhashed 1 00000080:00200000:1.0:1787827240.550719:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.550729:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.550738:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.550748:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.550755:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.550760:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.550763:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.550767:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.550771:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.550779:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.550789:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.550793:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.550798:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.550800:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.550803:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.550811:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub63, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.550815:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.550820:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.550823:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub63, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.550827:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub63, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.550886:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.550894:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a0b40 x1874671476547584/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.550910:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547584:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.550948:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.552035:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547584:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.552059:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x92:0x0](ffff9a789ddd6c08) 00000080:00200000:1.0:1787827240.552078:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub63, unhashed 1 00000080:00200000:1.0:1787827240.552107:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.552117:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.552128:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.552138:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.552146:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.552151:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.552154:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.552159:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.552163:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.552173:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.552184:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.552190:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.552196:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.552199:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.552203:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.552213:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub64, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.552217:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.552222:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.552226:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub64, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.552229:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub64, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.552258:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.552266:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a3840 x1874671476547840/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.552283:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a3840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547840:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.552325:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.553419:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a3840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476547840:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.553439:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x93:0x0](ffff9a789ddd42c8) 00000080:00200000:1.0:1787827240.553455:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub64, unhashed 1 00000080:00200000:1.0:1787827240.553477:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.553486:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.553496:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.553506:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.553531:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.553536:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.553539:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.553544:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.553548:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.553556:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.553566:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.553571:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.553576:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.553579:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.553582:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.553590:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub65, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.553594:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.553598:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.553602:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub65, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.553605:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub65, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.553626:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.553632:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a2940 x1874671476548096/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.553643:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a2940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548096:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.553673:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.554698:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a2940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548096:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.554725:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x94:0x0](ffff9a789ddd7448) 00000080:00200000:1.0:1787827240.554743:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub65, unhashed 1 00000080:00200000:1.0:1787827240.554771:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.554782:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.554792:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.554804:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.554812:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.554817:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.554821:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.554825:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.554866:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.554877:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.554887:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.554892:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.554897:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.554900:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.554903:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.554912:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub66, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.554916:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.554920:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.554924:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub66, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.554927:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub66, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.554954:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.554962:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a1680 x1874671476548352/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.554977:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a1680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548352:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.555018:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.556384:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a1680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548352:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.556433:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x95:0x0](ffff9a78aac00908) 00000080:00200000:1.0:1787827240.556453:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub66, unhashed 1 00000080:00200000:1.0:1787827240.556482:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.556493:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.556503:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.556531:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.556539:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.556544:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.556547:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.556552:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.556556:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.556566:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.556576:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.556582:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.556587:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.556591:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.556594:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.556604:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub67, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.556607:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.556611:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.556615:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub67, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.556618:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub67, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.556647:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.556654:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a0f00 x1874671476548608/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.556672:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a0f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548608:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.556716:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.558080:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548608:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.558104:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x96:0x0](ffff9a78aac063c8) 00000080:00200000:1.0:1787827240.558122:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub67, unhashed 1 00000080:00200000:1.0:1787827240.558147:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.558157:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.558166:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.558178:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.558186:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.558190:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.558193:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.558199:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.558202:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.558211:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.558221:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.558225:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.558229:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.558233:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.558236:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.558244:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub68, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.558248:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.558252:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.558256:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub68, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.558260:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub68, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.558284:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.558292:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a2d00 x1874671476548864/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.558308:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a2d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548864:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.558346:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.559666:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a2d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476548864:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.559688:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x97:0x0](ffff9a78aac04b08) 00000080:00200000:1.0:1787827240.559707:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub68, unhashed 1 00000080:00200000:1.0:1787827240.559735:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.559747:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.559757:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.559768:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.559777:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.559781:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.559784:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.559789:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.559793:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.559801:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.559811:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.559816:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.559820:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.559824:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.559827:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.559859:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub69, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.559862:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.559867:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.559871:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub69, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.559875:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub69, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.559902:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.559909:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65a30c0 x1874671476549120/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.559924:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a30c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549120:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.559981:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.561175:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a30c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549120:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.561204:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x98:0x0](ffff9a78aac03a88) 00000080:00200000:1.0:1787827240.561222:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub69, unhashed 1 00000080:00200000:1.0:1787827240.561249:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.561258:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.561269:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.561282:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.561290:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.561295:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.561298:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.561303:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.561307:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.561316:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.561326:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.561331:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.561336:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.561340:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.561343:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.561351:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub70, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.561355:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.561360:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.561363:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub70, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.561366:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub70, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.561409:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.561417:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645b840 x1874671476549376/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.561434:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645b840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549376:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.561477:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.562759:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645b840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549376:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.562775:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x99:0x0](ffff9a78aac01148) 00000080:00200000:1.0:1787827240.562788:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub70, unhashed 1 00000080:00200000:1.0:1787827240.562808:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.562816:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.562823:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.562858:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.562864:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.562867:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.562869:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.562873:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.562876:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.562882:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.562888:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.562891:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.562894:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.562896:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.562898:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.562905:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub71, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.562907:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.562910:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.562912:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub71, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.562915:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub71, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.562934:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.562939:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645a940 x1874671476549632/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.562951:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645a940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549632:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.562981:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.564174:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645a940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549632:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.564191:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x9a:0x0](ffff9a78aac000c8) 00000080:00200000:1.0:1787827240.564203:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub71, unhashed 1 00000080:00200000:1.0:1787827240.564221:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.564228:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.564234:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.564242:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.564258:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.564261:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.564263:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.564266:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.564268:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.564274:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.564280:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.564283:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.564285:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.564287:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.564288:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.564295:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub72, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.564297:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.564300:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.564302:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub72, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.564304:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub72, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.564321:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.564326:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6459680 x1874671476549888/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.564337:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6459680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549888:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.564366:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.565525:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6459680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476549888:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.565549:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x9b:0x0](ffff9a78aac021c8) 00000080:00200000:1.0:1787827240.565566:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub72, unhashed 1 00000080:00200000:1.0:1787827240.565590:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.565599:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.565607:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.565616:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.565621:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.565624:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.565626:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.565629:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.565631:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.565637:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.565643:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.565646:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.565648:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.565650:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.565652:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.565658:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub73, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.565660:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.565663:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.565665:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub73, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.565667:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub73, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.565685:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.565690:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6458f00 x1874671476550144/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.565700:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6458f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550144:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.565729:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.566906:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6458f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550144:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.566922:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x9c:0x0](ffff9a78aac03248) 00000080:00200000:1.0:1787827240.566935:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub73, unhashed 1 00000080:00200000:1.0:1787827240.566952:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.566959:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.566965:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.566972:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.566977:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.566980:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.566982:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.566985:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.566988:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.566993:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.567000:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.567002:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.567004:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.567006:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.567008:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.567013:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub74, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.567015:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.567018:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.567020:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub74, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.567022:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub74, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.567041:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.567048:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645ad00 x1874671476550400/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.567063:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645ad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550400:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.567101:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.568349:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645ad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550400:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.568369:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x9d:0x0](ffff9a78aac06c08) 00000080:00200000:1.0:1787827240.568382:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub74, unhashed 1 00000080:00200000:1.0:1787827240.568401:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.568408:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.568415:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.568422:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.568427:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.568431:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.568434:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.568439:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.568442:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.568450:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.568460:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.568464:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.568469:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.568472:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.568475:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.568484:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub75, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.568488:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.568492:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.568496:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub75, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.568500:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub75, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.568541:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.568549:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645b0c0 x1874671476550656/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.568564:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645b0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550656:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.568601:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.569898:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645b0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550656:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.569923:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x9e:0x0](ffff9a78aac042c8) 00000080:00200000:1.0:1787827240.569941:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub75, unhashed 1 00000080:00200000:1.0:1787827240.569984:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.569995:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.570005:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.570017:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.570026:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.570031:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.570034:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.570039:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.570043:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.570051:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.570061:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.570066:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.570071:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.570074:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.570077:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.570085:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub76, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.570088:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.570092:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.570096:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub76, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.570100:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub76, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.570127:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.570134:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645a1c0 x1874671476550912/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.570150:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645a1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550912:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.570189:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.571426:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645a1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476550912:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.571449:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0x9f:0x0](ffff9a78aac07448) 00000080:00200000:1.0:1787827240.571498:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub76, unhashed 1 00000080:00200000:1.0:1787827240.571542:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.571553:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.571563:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.571574:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.571582:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.571587:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.571590:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.571595:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.571599:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.571608:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.571618:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.571623:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.571627:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.571630:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.571633:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.571641:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub77, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.571645:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.571649:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.571653:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub77, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.571656:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub77, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.571680:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.571688:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a64592c0 x1874671476551168/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.571703:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a64592c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551168:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.571752:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.572815:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a64592c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551168:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.572851:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa0:0x0](ffff9a78aac05348) 00000080:00200000:1.0:1787827240.572865:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub77, unhashed 1 00000080:00200000:1.0:1787827240.572885:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.572893:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.572900:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.572908:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.572914:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.572917:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.572919:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.572922:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.572924:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.572930:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.572937:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.572939:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.572942:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.572944:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.572946:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.572951:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub78, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.572954:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.572957:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.572961:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub78, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.572965:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub78, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.572990:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.572998:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6458780 x1874671476551424/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.573014:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6458780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551424:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.573052:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.574201:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6458780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551424:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.574217:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa1:0x0](ffff9a78aac05b88) 00000080:00200000:1.0:1787827240.574230:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub78, unhashed 1 00000080:00200000:1.0:1787827240.574250:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.574260:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.574270:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.574282:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.574291:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.574295:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.574298:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.574303:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.574306:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.574312:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.574319:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.574322:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.574325:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.574327:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.574329:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.574335:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub79, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.574337:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.574340:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.574342:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub79, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.574344:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub79, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.574363:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.574368:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6459a40 x1874671476551680/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.574379:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6459a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551680:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.574409:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.575416:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6459a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551680:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.575440:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa2:0x0](ffff9a78aac02a08) 00000080:00200000:1.0:1787827240.575452:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub79, unhashed 1 00000080:00200000:1.0:1787827240.575469:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.575475:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.575482:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.575488:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.575493:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.575496:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.575498:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.575503:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.575516:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.575526:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.575537:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.575542:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.575546:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.575549:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.575552:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.575560:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub80, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.575563:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.575567:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.575570:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub80, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.575572:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub80, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.575597:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.575605:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6458000 x1874671476551936/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.575620:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6458000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551936:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.575659:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.576870:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6458000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476551936:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.576890:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa3:0x0](ffff9a78aac01988) 00000080:00200000:1.0:1787827240.576905:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub80, unhashed 1 00000080:00200000:1.0:1787827240.576930:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.576939:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.576947:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.576966:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.576973:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.576977:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.576980:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.576984:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.576987:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.576996:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.577005:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.577010:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.577013:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.577016:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.577019:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.577027:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub81, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.577030:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.577034:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.577037:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub81, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.577040:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub81, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.577063:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.577071:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a64583c0 x1874671476552192/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.577085:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a64583c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552192:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.577124:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.578221:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a64583c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552192:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.578266:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa4:0x0](ffff9a789dee00c8) 00000080:00200000:1.0:1787827240.578284:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub81, unhashed 1 00000080:00200000:1.0:1787827240.578307:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.578315:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.578325:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.578336:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.578344:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.578349:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.578351:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.578356:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.578359:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.578368:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.578377:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.578382:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.578386:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.578390:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.578393:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.578401:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub82, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.578405:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.578409:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.578412:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub82, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.578416:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub82, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.578450:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.578457:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645bc00 x1874671476552448/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.578472:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645bc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552448:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.578526:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.579653:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645bc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552448:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.579668:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa5:0x0](ffff9a789dee21c8) 00000080:00200000:1.0:1787827240.579680:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub82, unhashed 1 00000080:00200000:1.0:1787827240.579697:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.579704:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.579710:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.579717:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.579722:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.579733:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.579735:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.579738:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.579741:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.579746:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.579752:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.579755:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.579757:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.579759:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.579760:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.579765:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub83, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.579767:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.579770:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.579772:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub83, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.579775:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub83, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.579792:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.579797:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645b480 x1874671476552704/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.579807:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645b480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552704:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.579849:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.580827:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645b480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552704:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.580879:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa6:0x0](ffff9a789dee3248) 00000080:00200000:1.0:1787827240.580897:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub83, unhashed 1 00000080:00200000:1.0:1787827240.580916:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.580924:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.580930:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.580938:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.580944:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.580947:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.580949:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.580959:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.580962:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.580970:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.580977:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.580980:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.580983:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.580985:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.580987:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.580993:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub84, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.580995:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.580998:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.581000:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub84, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.581002:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub84, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.581020:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.581026:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6459e00 x1874671476552960/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.581041:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6459e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552960:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.581077:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.582022:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6459e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476552960:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.582042:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa7:0x0](ffff9a789dee6c08) 00000080:00200000:1.0:1787827240.582059:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub84, unhashed 1 00000080:00200000:1.0:1787827240.582083:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.582093:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.582100:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.582107:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.582112:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.582115:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.582117:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.582120:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.582122:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.582128:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.582134:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.582137:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.582139:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.582141:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.582143:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.582148:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub85, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.582151:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.582153:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.582155:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub85, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.582158:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub85, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.582176:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.582181:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a645a580 x1874671476553216/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.582191:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645a580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553216:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.582237:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.583269:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645a580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553216:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.583292:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa8:0x0](ffff9a789dee42c8) 00000080:00200000:1.0:1787827240.583310:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub85, unhashed 1 00000080:00200000:1.0:1787827240.583334:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.583345:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.583354:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.583364:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.583372:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.583376:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.583379:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.583385:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.583388:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.583396:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.583406:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.583410:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.583414:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.583418:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.583421:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.583428:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub86, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.583432:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.583437:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.583440:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub86, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.583443:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub86, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.583468:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.583476:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6458b40 x1874671476553472/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.583490:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6458b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553472:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.583540:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.584651:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6458b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553472:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.584670:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xa9:0x0](ffff9a789dee7448) 00000080:00200000:1.0:1787827240.584682:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub86, unhashed 1 00000080:00200000:1.0:1787827240.584700:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.584707:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.584713:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.584720:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.584725:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.584728:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.584730:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.584733:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.584735:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.584741:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.584746:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.584749:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.584751:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.584753:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.584754:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.584759:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub87, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.584761:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.584764:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.584766:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub87, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.584768:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub87, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.584797:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.584802:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1e940 x1874671476553728/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.584812:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553728:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.584857:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.586092:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553728:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.586113:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xaa:0x0](ffff9a789dee5348) 00000080:00200000:1.0:1787827240.586128:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub87, unhashed 1 00000080:00200000:1.0:1787827240.586150:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.586158:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.586167:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.586177:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.586185:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.586189:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.586192:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.586197:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.586200:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.586210:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.586219:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.586222:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.586225:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.586227:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.586229:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.586235:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub88, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.586237:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.586240:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.586242:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub88, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.586244:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub88, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.586262:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.586267:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1d680 x1874671476553984/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.586277:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553984:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.586306:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.587368:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476553984:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.587388:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xab:0x0](ffff9a789dee5b88) 00000080:00200000:1.0:1787827240.587404:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub88, unhashed 1 00000080:00200000:1.0:1787827240.587421:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.587427:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.587434:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.587440:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.587445:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.587448:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.587450:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.587453:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.587456:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.587461:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.587467:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.587470:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.587472:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.587474:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.587475:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.587480:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub89, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.587482:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.587485:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.587487:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub89, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.587489:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub89, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.587505:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.587523:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1cf00 x1874671476554240/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.587533:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476554240:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.587562:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.588699:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476554240:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.588714:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xac:0x0](ffff9a789dee2a08) 00000080:00200000:1.0:1787827240.588726:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub89, unhashed 1 00000080:00200000:1.0:1787827240.588742:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.588749:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.588755:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.588762:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.588766:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.588769:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.588771:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.588774:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.588776:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.588782:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.588788:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.588790:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.588792:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.588794:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.588795:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.588801:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub90, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.588803:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.588805:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.588807:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub90, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.588809:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub90, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.588825:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.588845:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1ed00 x1874671476554496/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.588858:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1ed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476554496:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.588894:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.590048:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1ed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476554496:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.590069:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xad:0x0](ffff9a789dee1988) 00000080:00200000:1.0:1787827240.590084:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub90, unhashed 1 00000080:00200000:1.0:1787827240.590106:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.590116:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.590124:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.590135:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.590142:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.590147:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.590149:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.590154:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.590157:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.590166:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.590175:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.590180:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.590184:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.590187:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.590190:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.590198:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub91, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.590201:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.590205:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.590209:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub91, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.590212:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub91, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.590235:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.590242:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1f0c0 x1874671476554752/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.590256:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476554752:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.590291:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.591319:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476554752:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.591341:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xae:0x0](ffff9a789dee0908) 00000080:00200000:1.0:1787827240.591359:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub91, unhashed 1 00000080:00200000:1.0:1787827240.591380:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.591390:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.591398:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.591408:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.591416:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.591421:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.591423:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.591429:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.591432:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.591440:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.591449:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.591454:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.591459:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.591462:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.591464:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.591472:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub92, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.591476:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.591480:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.591483:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub92, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.591487:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub92, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.591523:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.591531:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1e1c0 x1874671476555008/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.591546:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555008:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.591580:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.592653:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555008:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.592674:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xaf:0x0](ffff9a789dee63c8) 00000080:00200000:1.0:1787827240.592712:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub92, unhashed 1 00000080:00200000:1.0:1787827240.592738:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.592748:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.592758:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.592769:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.592777:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.592782:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.592785:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.592789:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.592793:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.592801:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.592811:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.592815:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.592819:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.592823:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.592825:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.592857:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub93, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.592861:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.592867:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.592870:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub93, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.592874:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub93, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.592901:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.592908:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1d2c0 x1874671476555264/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.592922:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555264:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.592986:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.594196:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555264:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.594219:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xb0:0x0](ffff9a789dee4b08) 00000080:00200000:1.0:1787827240.594236:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub93, unhashed 1 00000080:00200000:1.0:1787827240.594259:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.594269:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.594278:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.594288:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.594297:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.594302:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.594305:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.594310:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.594314:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.594324:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.594334:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.594339:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.594344:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.594347:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.594350:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.594359:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub94, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.594363:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.594367:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.594371:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub94, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.594375:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub94, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.594402:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.594409:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1c780 x1874671476555520/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.594425:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555520:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.594465:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.595647:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555520:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.595667:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xb1:0x0](ffff9a789dee3a88) 00000080:00200000:1.0:1787827240.595684:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub94, unhashed 1 00000080:00200000:1.0:1787827240.595705:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.595715:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.595724:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.595735:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.595743:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.595748:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.595751:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.595756:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.595760:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.595768:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.595778:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.595783:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.595787:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.595791:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.595793:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.595802:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub95, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.595805:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.595810:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.595813:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub95, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.595817:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub95, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.595863:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.595871:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1da40 x1874671476555776/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.595886:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555776:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.595922:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.597038:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476555776:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.597059:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xb2:0x0](ffff9a789dee1148) 00000080:00200000:1.0:1787827240.597074:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub95, unhashed 1 00000080:00200000:1.0:1787827240.597096:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.597107:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.597115:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.597126:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.597134:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.597139:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.597142:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.597147:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.597151:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.597160:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.597170:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.597175:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.597179:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.597182:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.597185:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.597195:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub96, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.597198:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.597203:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.597206:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub96, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.597210:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub96, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.597236:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.597244:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1c000 x1874671476556032/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.597259:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556032:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.597298:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.598578:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556032:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.598617:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xb3:0x0](ffff9a789deeaa08) 00000080:00200000:1.0:1787827240.598648:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub96, unhashed 1 00000080:00200000:1.0:1787827240.598673:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.598683:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.598693:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.598705:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.598713:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.598718:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.598721:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.598726:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.598730:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.598739:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.598758:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.598763:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.598768:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.598771:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.598775:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.598785:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub97, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.598789:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.598793:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.598796:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub97, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.598799:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub97, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.598826:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.598869:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1c3c0 x1874671476556288/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.598887:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556288:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.598945:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.601130:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556288:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.601174:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xb4:0x0](ffff9a789dee9988) 00000080:00200000:1.0:1787827240.601203:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub97, unhashed 1 00000080:00200000:1.0:1787827240.601246:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.601257:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.601268:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.601278:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.601283:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.601287:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.601289:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.601295:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.601299:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.601307:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.601320:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.601326:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.601331:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.601334:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.601337:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.601349:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub98, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.601352:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.601356:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.601366:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub98, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.601371:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub98, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.601405:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.601410:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1fc00 x1874671476556544/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.601424:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556544:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.601464:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.602657:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556544:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.602673:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xb5:0x0](ffff9a789dee8908) 00000080:00200000:1.0:1787827240.602684:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub98, unhashed 1 00000080:00200000:1.0:1787827240.602702:0:48375:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.602709:0:48375:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.602715:0:48375:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.602722:0:48375:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.602726:0:48375:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.602729:0:48375:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.602731:0:48375:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.602734:0:48375:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e2bbd0 00010000:00010000:1.0:1787827240.602736:0:48375:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.602742:0:48375:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.602748:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.602759:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:1.0:1787827240.602765:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.602767:0:48375:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds1, flags=24 00000080:00200000:1.0:1787827240.602768:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.602774:0:48375:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub99, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=1536 00000080:00200000:1.0:1787827240.602776:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.602779:0:48375:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), inode mode 41ed mask 3 00000080:00200000:1.0:1787827240.602781:0:48375:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub99, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8), flags=33060, excl=1 00000080:00200000:1.0:1787827240.602783:0:48375:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub99, dir=[0x200001b72:0x2:0x0](ffff9a789ddb42c8) mode 100444 dev 0 00000020:00100000:1.0:1787827240.602800:0:48375:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787827240.602805:0:48375:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890b1f480 x1874671476556800/t0(0) o36->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827240.602815:0:48375:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556800:192.168.203.140@tcp:36:createmany.0 00000100:00100000:1.0:1787827240.602893:0:48375:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78913933c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.605106:0:48375:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48375:1874671476556800:192.168.203.140@tcp:36:createmany.0 00000080:00200000:1.0:1787827240.605149:0:48375:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200001b72:0xb6:0x0](ffff9a789deee3c8) 00000080:00200000:1.0:1787827240.605178:0:48375:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub99, unhashed 1 00000080:00200000:1.0:1787827240.624302:0:48380:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827240.624331:0:48380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827240.624351:0:48380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.624367:0:48380:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.624377:0:48380:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827240.624386:0:48380:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827240.624390:0:48380:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827240.624400:0:48380:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e63b78 00010000:00010000:1.0:1787827240.624405:0:48380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827240.624417:0:48380:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827240.624436:0:48380:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:1.0:1787827240.624443:0:48380:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=13 00000080:00200000:1.0:1787827240.624452:0:48380:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9a78a6512000) d6.sanity-scrub proj=0 00000080:00000004:1.0:1787827240.624456:0:48380:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9a78a6512000: age 1611, max_age 1612 00800000:00000004:1.0:1787827240.624467:0:48380:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9a78a6512000: age 1611, max_age 1612 00000100:00100000:1.0:1787827240.624531:0:48380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job stat:97196825-2cb9-49ba-a47d-1455a997ce3a:48380:1874671476556928:192.168.203.140@tcp:41:stat.0 00000100:00100000:1.0:1787827240.624622:0:48380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78813efe40 going to sleep for 11 seconds 00000100:00100000:1.0:1787827240.626188:0:48380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job stat:97196825-2cb9-49ba-a47d-1455a997ce3a:48380:1874671476556928:192.168.203.140@tcp:41:stat.0 00800000:00000004:1.0:1787827240.626209:0:48380:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9a78a6512000: update ffff9a7886c89e08 cache blocks 139250/156552 objects 199104/200008 00000080:00000004:1.0:1787827240.626215:0:48380:0:(obd_class.h:1116:obd_statfs()) lustre-clilmv-ffff9a78a6512000: update ffff9a7886c88450 cache blocks 139250/156552 objects 199104/200008 00000080:00000004:1.0:1787827240.626219:0:48380:0:(llite_lib.c:2755:ll_statfs_internal()) MDC blocks 139250/156552 objects 199104/200008 00010000:00010000:1.0:1787827240.649952:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.649963:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504200/0xebdcd48a4298e49d lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x4059aa0aef8db180 expref: -99 pid: 48384 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.649975:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7881504200 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:1.0:1787827240.649981:0:47597:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504200/0xebdcd48a4298e49d lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x4059aa0aef8db180 expref: -99 pid: 48384 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.649992:0:47597:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504200/0xebdcd48a4298e49d lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x4059aa0aef8db180 expref: -99 pid: 48384 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.650002:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504200/0xebdcd48a4298e49d lrc: 3/0,0 mode: PR/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x4059aa0aef8db180 expref: -99 pid: 48384 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.650029:0:47597:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890939200/0xebdcd48a4298d9d7 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8fc2 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827240.650065:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504200/0xebdcd48a4298e49d lrc: 2/0,0 mode: --/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x4059aa0aef8db180 expref: -99 pid: 48384 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.650074:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.650103:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504200/0xebdcd48a4298e49d lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x4059aa0aef8db180 expref: -99 pid: 48384 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.650115:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504200/0xebdcd48a4298e49d lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x4059aa0aef8db180 expref: -99 pid: 48384 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.650138:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.650140:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.650142:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.679033:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.679045:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e4c7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x82:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db228 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.679055:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7881505000 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.679061:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e4c7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x82:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db228 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.679075:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.679090:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.679116:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e4c7 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x82:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db228 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.679122:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.679146:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e4c7 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x82:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db228 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.679158:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e4c7 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x82:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db228 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.679171:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.679174:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.679177:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.683351:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.683359:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e4d5 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x4059aa0aef8db24b expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.683368:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7884b12400 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:1.0:1787827240.683373:0:47597:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e4d5 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x4059aa0aef8db24b expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.683385:0:47597:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e4d5 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x4059aa0aef8db24b expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.683396:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e4d5 lrc: 3/0,0 mode: PR/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8db24b expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.683412:0:47597:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504400/0xebdcd48a4298e4ab lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db1cd expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.683439:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e4d5 lrc: 2/0,0 mode: --/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8db24b expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.683445:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.683469:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e4d5 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8db24b expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.683477:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e4d5 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8db24b expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.683487:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.683490:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.683494:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.689953:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a57f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476560640:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.695303:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.707299:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.707541:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.707552:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000950c25eb/0xebdcd48a4298e51b lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x83:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db308 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.707565:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890ba2800 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.707573:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000950c25eb/0xebdcd48a4298e51b lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x83:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db308 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.707592:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.707603:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.707640:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000950c25eb/0xebdcd48a4298e51b lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x83:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db308 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.707650:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.707674:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000950c25eb/0xebdcd48a4298e51b lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x83:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db308 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.707685:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000950c25eb/0xebdcd48a4298e51b lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x83:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db308 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.707707:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.707712:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.707715:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.720525:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a57e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476562944:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.721440:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 6/2,0 mode: --/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.721452:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 6/2,0 mode: --/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.721464:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.721473:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.721483:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.721491:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827240.721500:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.721519:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.721525:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.721532:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e53e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8db363 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827240.721541:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a57e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476562944:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.723044:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787827240.723054:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a57f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476563200:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827240.724939:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a57f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476563200:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827240.733669:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.733898:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.733909:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e561 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x84:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db3be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.733922:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7881505000 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.733929:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e561 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x84:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db3be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.733949:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.733961:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.733994:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e561 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x84:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db3be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.734004:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.734025:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e561 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x84:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db3be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.734035:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b9d44460/0xebdcd48a4298e561 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x84:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db3be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.734053:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.734058:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.734061:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.746058:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a64a9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476565248:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.750570:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.762343:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.769485:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a64a9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476567552:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.772047:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787827240.772061:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a57e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476567808:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827240.773040:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.774002:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a57e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476567808:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827240.782420:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.782708:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.782718:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a07a712f/0xebdcd48a4298e5ed lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x86:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db52a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.782729:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7888719c00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.782735:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a07a712f/0xebdcd48a4298e5ed lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x86:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db52a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.782751:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.782760:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.782794:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a07a712f/0xebdcd48a4298e5ed lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x86:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db52a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.782803:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.782850:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a07a712f/0xebdcd48a4298e5ed lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x86:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db52a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.782858:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a07a712f/0xebdcd48a4298e5ed lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x86:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db52a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.782877:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.782880:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.782885:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.790307:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476569856:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.792622:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787827240.792635:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a645a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476570112:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827240.793478:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.794256:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a645a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476570112:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827240.801698:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.801913:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.801924:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000033eda377/0xebdcd48a4298e633 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db5e0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.801937:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78a64e1800 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.801943:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000033eda377/0xebdcd48a4298e633 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db5e0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.801961:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.801970:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.801997:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000033eda377/0xebdcd48a4298e633 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db5e0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.802004:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.802022:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000033eda377/0xebdcd48a4298e633 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db5e0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.802029:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000033eda377/0xebdcd48a4298e633 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x87:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db5e0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.802039:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.802041:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.802044:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.809194:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a64583c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476572160:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.809990:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.810007:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.810021:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.810033:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.810046:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.810056:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827240.810069:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.810080:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.810094:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.810118:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298e656 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8db63b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827240.810147:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a64583c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476572160:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.820864:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.821033:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.821041:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006c9977ab/0xebdcd48a4298e679 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db696 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.821050:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7883551600 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.821055:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006c9977ab/0xebdcd48a4298e679 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db696 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.821067:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.821074:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.821094:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006c9977ab/0xebdcd48a4298e679 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db696 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.821100:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.821115:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006c9977ab/0xebdcd48a4298e679 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db696 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.821121:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006c9977ab/0xebdcd48a4298e679 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db696 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.821130:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.821132:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.821135:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.829458:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838ce580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476574464:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.830434:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 6/2,0 mode: --/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.830445:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 6/2,0 mode: --/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.830459:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.830467:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.830475:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.830482:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827240.830490:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.830497:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.830502:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.830531:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827240.830553:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838ce580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476574464:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.831781:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.831788:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.831797:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7888718400 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.831801:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.831812:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.831818:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.831860:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.831867:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.831886:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.831892:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e69c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x88:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db6f1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.831908:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.831911:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.831913:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.841056:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.841243:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.841251:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac607369/0xebdcd48a4298e6bf lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db74c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.841260:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7883551e00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.841265:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac607369/0xebdcd48a4298e6bf lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db74c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.841277:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x10:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.841284:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x10:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.841306:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac607369/0xebdcd48a4298e6bf lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db74c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.841312:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.841328:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac607369/0xebdcd48a4298e6bf lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db74c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.841334:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac607369/0xebdcd48a4298e6bf lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db74c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.841345:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.841349:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.841351:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.849883:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.849896:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.849910:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.849923:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.849935:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.849946:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827240.849959:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.849966:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.849972:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.849983:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827240.849994:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6458f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476576768:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.851119:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.851274:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.851281:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.851289:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7883550e00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.851294:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.851305:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x11:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.851312:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x11:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.851342:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.851351:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.851375:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.851385:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e6e2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x89:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db7a7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.851409:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.851413:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.851415:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.853330:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476577024:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827240.861360:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.861599:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.861608:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ae699d54/0xebdcd48a4298e705 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db802 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.861617:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7883551a00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.861623:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ae699d54/0xebdcd48a4298e705 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db802 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.861637:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x12:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.861645:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x12:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.861670:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ae699d54/0xebdcd48a4298e705 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db802 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.861676:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.861693:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ae699d54/0xebdcd48a4298e705 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db802 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.861699:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ae699d54/0xebdcd48a4298e705 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db802 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.861715:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.861717:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.861718:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.870614:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6459680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476579072:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.871649:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.871662:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.871673:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.871681:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.871690:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.871698:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827240.871707:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.871714:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.871720:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.871726:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827240.871735:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6459680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476579072:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.873165:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.873176:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.873189:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7883550a00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.873196:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.873215:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x13:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.873225:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x13:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.873262:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.873272:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.873301:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.873334:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000080b46d99/0xebdcd48a4298e728 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x8a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db85d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.873351:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.873354:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.873357:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.884092:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827240.884337:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.884346:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e74b lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x8b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8db8b8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.884356:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7888718400 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.884362:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e74b lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x8b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8db8b8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.884378:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x14:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.884386:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x14:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.884415:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e74b lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x8b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db8b8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.884422:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.884439:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e74b lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x8b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db8b8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.884446:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000010be268d/0xebdcd48a4298e74b lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x8b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8db8b8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.884460:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.884462:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.884464:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.895815:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.906565:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.916308:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.927722:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.933736:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476585984:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.936464:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.944479:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.949718:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838ccb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476588288:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.952676:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.953433:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476588544:192.168.203.140@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787827240.961014:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827240.961024:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000de8e8b19/0xebdcd48a4298e863 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x8f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dbb90 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.961034:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7886b37800 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827240.961040:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000de8e8b19/0xebdcd48a4298e863 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x8f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dbb90 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827240.961058:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x1c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827240.961071:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x1c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827240.961112:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000de8e8b19/0xebdcd48a4298e863 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x8f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbb90 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.961121:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827240.961153:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000de8e8b19/0xebdcd48a4298e863 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x8f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbb90 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.961163:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000de8e8b19/0xebdcd48a4298e863 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x8f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbb90 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827240.961182:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.961186:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.961188:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827240.968434:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a668f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476590592:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827240.969155:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.969172:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.969186:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.969200:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827240.969217:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827240.969224:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827240.969234:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.969241:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.969247:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827240.969253:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000feed6033/0xebdcd48a4298e886 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dbbeb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827240.969264:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a668f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476590592:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.980630:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.986743:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66103c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476592896:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827240.989882:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827240.998746:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.009769:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827241.010050:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.010059:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054a7c12e/0xebdcd48a4298e912 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x91:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dbd57 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.010068:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890978e00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.010073:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054a7c12e/0xebdcd48a4298e912 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x91:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dbd57 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.010089:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x21:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.010097:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x21:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.010125:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054a7c12e/0xebdcd48a4298e912 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x91:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbd57 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.010139:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.010162:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054a7c12e/0xebdcd48a4298e912 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x91:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbd57 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.010174:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054a7c12e/0xebdcd48a4298e912 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x91:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbd57 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.010196:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.010199:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.010203:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827241.020146:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.021406:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827241.030011:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.030036:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.030053:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.030065:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.030080:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.030093:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.030107:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.030116:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.030126:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.030151:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.030189:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476597504:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827241.031215:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827241.031401:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.031411:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.031423:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890979a00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.031430:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x92:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.031447:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x23:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.031456:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x23:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.031490:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x92:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.031500:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.031542:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x92:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.031552:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c4240dc4/0xebdcd48a4298e958 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x92:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dbe0d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.031575:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.031580:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.031584:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827241.039959:0:10935:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787827241.039976:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476598656:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.040776:0:10935:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.040787:0:10935:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.040797:0:10935:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.040805:0:10935:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.040814:0:10935:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 5/2,0 mode: PR/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.040822:0:10935:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.040848:0:10935:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.040859:0:10935:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.040869:0:10935:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.040880:0:10935:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bc0fc067/0xebdcd48a4298e97b lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dbe68 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.040894:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476598656:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827241.042167:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476598912:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.050648:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6611a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476599808:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827241.053974:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.063814:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.065118:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.071184:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476602112:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827241.073005:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.073263:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787827241.073275:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476602368:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.074091:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.075018:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476602368:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.083042:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.083398:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476603520:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.093697:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.102465:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476605824:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.103257:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.104048:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476605824:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.119419:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476608128:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.120220:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.125929:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6417c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476609024:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827241.128770:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.138427:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.144325:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6414f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476611328:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827241.147260:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.155456:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.155857:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6415e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476612736:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.157616:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6415e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476612736:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.163751:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6417840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476613632:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.164590:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.164603:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.164614:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.164621:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.164632:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.164640:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.164649:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.164656:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.164661:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.164667:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb42 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x99:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dc307 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.164675:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6417840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476613632:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.175682:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.175692:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb65 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x9a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc362 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.175703:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78a65c7c00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.175709:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x9a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc362 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.175725:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x32:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.175733:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x32:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.175758:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb65 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x9a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc362 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.175764:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.175786:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb65 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x9a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc362 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.175796:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c29da0ce/0xebdcd48a4298eb65 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x9a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc362 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.175814:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.175819:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.175823:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827241.183518:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6416580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476615936:192.168.203.140@tcp:101:cp.0 00000100:00100000:1.0:1787827241.185634:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787827241.185649:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6416580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476616192:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.186409:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.195164:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476617344:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.196117:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.202768:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6673480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476618240:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.214113:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.214122:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000908e9c73/0xebdcd48a4298ebf1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x9c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc4ce expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.214131:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7881639200 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.214136:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000908e9c73/0xebdcd48a4298ebf1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x9c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc4ce expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.214149:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x36:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.214157:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x36:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.214184:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000908e9c73/0xebdcd48a4298ebf1 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x9c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc4ce expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.214190:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.214207:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000908e9c73/0xebdcd48a4298ebf1 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x9c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc4ce expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.214213:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000908e9c73/0xebdcd48a4298ebf1 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x9c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc4ce expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.214224:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.214225:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.214227:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827241.221267:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476620544:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.221999:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.222016:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.222027:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.222034:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.222042:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.222049:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.222058:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.222064:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.222070:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.222076:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000078c8d01b/0xebdcd48a4298ec14 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x9c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dc529 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.222104:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476620544:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.233424:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.233433:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007dd246fa/0xebdcd48a4298ec37 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc584 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.233442:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78c21da000 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.233446:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007dd246fa/0xebdcd48a4298ec37 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc584 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.233459:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x38:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.233467:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x38:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.233491:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007dd246fa/0xebdcd48a4298ec37 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc584 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.233497:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.233529:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007dd246fa/0xebdcd48a4298ec37 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc584 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.233537:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007dd246fa/0xebdcd48a4298ec37 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc584 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.233549:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.233551:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.233553:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1787827241.234815:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x38:0x0] updating i_size 6821 i_blocks 0 00000080:00200000:2.0:1787827241.234819:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 6821 00000080:00200008:2.0:1787827241.234852:0:48385:0:(file.c:479:ll_file_release()) START file sanity-dom.sh:[0x240001b71:0x38:0x0](ffff9a789dfc1988), flags 100001 00000002:00100000:2.0:1787827241.234886:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6672580 x1874671476621568/t17179869483(17179869483) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787827241.234898:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787827241.234904:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6670b40 x1874671476622208/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787827241.234920:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6670b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622208:192.168.203.140@tcp:35:cp.0 00000100:00100000:2.0:1787827241.234960:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27540 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.234989:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476621952:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.235623:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6670b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622208:192.168.203.140@tcp:35:cp.0 00000002:00100000:2.0:1787827241.235638:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6672580 x1874671476621568/t17179869483(17179869483) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.235668:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.235676:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.235683:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.235692:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.235697:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.235700:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.235702:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.235706:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.235709:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.235719:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.235726:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.235729:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.235732:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.235734:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.235736:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.235738:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=sanity-dom.sh, flags=8 00000080:00200008:2.0:1787827241.235742:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-dom.sh:[0x240001b71:0x38:0x0](ffff9a789dfc1988), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.235745:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x38:0x0](ffff9a789dfc1988),name=sanity-dom.sh 00000002:00010000:2.0:1787827241.235751:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x38:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.235755:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21dbe00/0xebdcd48a4298ec30 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc576 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.235761:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21dbe00/0xebdcd48a4298ec30 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc576 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.235766:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.235768:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x38:0x0](ffff9a789dfc1988) 00000080:00010000:2.0:1787827241.235769:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x38:0x0](ffff9a789dfc1988) for lock 0xebdcd48a4298ec30 00000080:00010000:2.0:1787827241.235771:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ec30 from it ffffaa5487e7bcb8 00010000:00010000:2.0:1787827241.235773:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21dbe00/0xebdcd48a4298ec30 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc576 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.235779:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21dbe00/0xebdcd48a4298ec30 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc576 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.235788:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x38:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.235792:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x38:0x0] 00000008:00010000:2.0:1787827241.235798:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.235800:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7883461fb0, osc lock ffff9a78840629c0, flags 0x400001002 00010000:00010000:2.0:1787827241.235806:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000fc71458b/0xebdcd48a4298ec3e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc58b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.235813:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000fc71458b/0xebdcd48a4298ec3e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc58b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:2.0:1787827241.235820:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:2.0:1787827241.235822:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000fc71458b/0xebdcd48a4298ec3e lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc58b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.235847:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000fc71458b/0xebdcd48a4298ec3e lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc58b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:2.0:1787827241.235859:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1787827241.235864:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x38:0x0] updating i_size 6821 i_blocks 0 00000020:00010000:2.0:1787827241.235868:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.235871:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.235875:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7883461fb0 (ffff9a78899dba28/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.235878:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.235881:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7883461fb0 (ffff9a78899db488/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.235889:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x38:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.235919:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.235926:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.235932:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.235939:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.235944:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.235947:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.235949:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.235952:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.235954:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.235959:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.235965:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.235967:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.235970:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.235972:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.235973:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.235980:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-ec.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:2.0:1787827241.235983:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-ec.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.235988:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-ec.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.236006:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236011:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236017:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.236027:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66712c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622336:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.236059:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27540 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.236893:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66712c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622336:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.236900:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236907:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236912:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236916:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236921:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236924:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a78c21da400 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.236926:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236930:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236934:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236939:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.236942:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.236946:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da400/0xebdcd48a4298ec45 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.236954:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a66712c0 x1874671476622336/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.236961:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.236970:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.236988:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.236993:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.236998:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.237004:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.237009:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.237012:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.237014:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.237016:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.237018:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.237023:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.237029:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.237032:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.237035:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.237037:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.237039:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.237044:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-ec.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:2.0:1787827241.237047:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-ec.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.237051:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-ec.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.237067:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237072:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237077:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.237085:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66712c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622464:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.237114:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27540 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.237896:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66712c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622464:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.237903:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237910:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237914:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237919:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237924:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237927:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890906400 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.237929:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237933:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237937:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237941:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.237944:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.237948:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906400/0xebdcd48a4298ec4c lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.237956:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a66712c0 x1874671476622464/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.237963:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.237971:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.238002:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.238008:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.238012:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.238019:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.238024:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.238027:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.238029:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.238032:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:2.0:1787827241.238034:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.238039:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.238045:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.238048:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.238050:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.238052:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.238054:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.238060:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.238063:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=sanity-ec.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78894b2a80, open_flags 80c1, mode 81a4 opened 0 00000080:00200000:2.0:1787827241.238067:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-ec.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:2.0:1787827241.238074:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-ec.sh,[0x240001b71:0x39:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:2.0:1787827241.238091:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.238096:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.238102:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.238107:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66712c0 x1874671476622720/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.238113:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.238122:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66712c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622720:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.238186:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27540 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.239189:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66712c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622720:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.239198:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239208:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239214:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239220:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239225:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239231:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239240:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239245:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.239251:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a66712c0 x1874671476622720/t17179869485(17179869485) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.239257:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a66712c0 x1874671476622720/t17179869485(17179869485) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.239264:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.239270:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239275:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/57 (0 0) 00000080:00200000:2.0:1787827241.239278:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.239282:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=sanity-ec.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:2.0:1787827241.239294:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x39:0x0](ffff9a789dfc0908) 00000080:00200000:2.0:1787827241.239323:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x39:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.239326:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x39:0x0](ffff9a789dfc0908) 00000080:00010000:2.0:1787827241.239328:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x39:0x0](ffff9a789dfc0908) for lock 0xebdcd48a4298ec53 00000080:00010000:2.0:1787827241.239333:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x39:0x0](ffff9a789dfc0908) 00000080:00200008:2.0:1787827241.239338:0:48385:0:(file.c:1090:ll_file_open()) START file sanity-ec.sh:[0x240001b71:0x39:0x0](ffff9a789dfc0908), flags 100301 00000002:00100000:2.0:1787827241.239347:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a66712c0 x1874671476622720/t17179869485(17179869485) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.239366:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ec53 from it ffff9a78877da960 00010000:00010000:2.0:1787827241.239370:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239376:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.239385:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x39:0x0](ffff9a789dfc0908), inode mode 81a4 mask 40 00000080:00200008:2.0:1787827241.239393:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-ec.sh:[0x240001b71:0x39:0x0](ffff9a789dfc0908), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.239395:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x39:0x0](ffff9a789dfc0908),name=sanity-ec.sh 00000002:00010000:2.0:1787827241.239402:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x39:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.239408:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.239414:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.239419:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.239421:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x39:0x0](ffff9a789dfc0908) 00000080:00010000:2.0:1787827241.239423:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x39:0x0](ffff9a789dfc0908) for lock 0xebdcd48a4298ec53 00000080:00010000:2.0:1787827241.239425:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ec53 from it ffffaa5487e7bcd8 00010000:00010000:2.0:1787827241.239428:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.239433:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.239442:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x39:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.239445:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x39:0x0] 00000008:00010000:2.0:1787827241.239451:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.239453:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7883461eb0, osc lock ffff9a78840629c0, flags 0x400001002 00010000:00010000:2.0:1787827241.239471:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.239477:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1787827241.239498:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.239527:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476622848:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.240322:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.240334:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.240343:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.240350:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.240358:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.240365:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.240374:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.240381:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.240386:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.240394:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.240398:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x39:0x0] updating i_size 0 i_blocks 0 00000100:00100000:1.0:1787827241.240404:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476622848:192.168.203.140@tcp:101:cp.0 00000020:00010000:2.0:1787827241.240404:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.240410:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.240413:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7883461eb0 (ffff9a78899dba28/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.240415:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.240418:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7883461eb0 (ffff9a78899db488/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.240426:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x39:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.240517:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-ec.sh:[0x240001b71:0x39:0x0], ppos: 0, count: 123804 00000080:00200000:2.0:1787827241.240523:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-ec.sh: write ppos: 0, bytes: 123804 00000080:00200000:2.0:1787827241.240525:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-ec.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.240529:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000a001e66e[0x110, 1, [0x240001b71:0x39:0x0] hash] 00000020:00200000:2.0:1787827241.240531:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 123804) 0 0 00000080:00200000:2.0:1787827241.240533:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x39:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.240538:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 30] 00020000:00200000:2.0:1787827241.240542:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 123804) 123804, 123804 00020000:00200000:2.0:1787827241.240544:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.240550:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 30] 00000008:00010000:2.0:1787827241.240555:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.240558:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7883461eb0, osc lock ffff9a78840629c0, flags 0x0 00010000:00010000:2.0:1787827241.240563:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419264/157 (0 126975) 00010000:00010000:2.0:1787827241.240579:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.240585:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->126975] (req 0->126975) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.240591:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->126975] (req 0->126975) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.240600:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622976:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.240629:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27000 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.241231:0:47593:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488782720 00000100:00100000:2.0:1787827241.241236:0:47593:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488782720 00000100:00100000:2.0:1787827241.241262:0:47593:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 267 00000100:00100000:2.0:1787827241.241268:0:47593:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c8780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:82310:x1874671488782720:12345-192.168.203.140@tcp:104: 00010000:00010000:2.0:1787827241.241275:0:47593:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ec5a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x9d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc5df expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.241307:0:47593:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787827241.241311:0:47593:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c8780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:82310:x1874671488782720:12345-192.168.203.140@tcp:104: Request processed in 45us (112us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.241316:0:47593:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 267 00000100:00100000:2.0:1787827241.241914:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476622976:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.241927:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->126975] (req 0->126975) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.241939:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->126975] (req 0->126975) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.241949:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->126975] (req 0->126975) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.242299:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488782848 00000100:00100000:2.0:1787827241.242303:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488782848 00000100:00100000:2.0:1787827241.242326:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 268 00000100:00100000:2.0:1787827241.242331:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a65a2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488782848:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.242338:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->126975] (req 0->126975) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.242344:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->126975] (req 0->126975) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.242351:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->126975) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.242356:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->126975) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.242361:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->126975) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.242371:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000077e971a6/0xebdcd48a4298ec61 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0x9d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->126975) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc5e6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.242384:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890907200) 00000100:00100000:2.0:1787827241.242411:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a65a2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488782848:12345-192.168.203.140@tcp:105: Request processed in 77us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.242426:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 268 00000100:00100000:2.0:1787827241.242914:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476623104:192.168.203.140@tcp:103:ldlm_bl.0 00000080:00200000:2.0:1787827241.249390:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3a:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.249396:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.249398:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.249401:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a789081d630 (ffff9a78899dba28/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.249404:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890933160 (ffff9a78899db488/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.249406:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a789081d630 (ffff9a78899db488/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.249413:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3a:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.249478:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-flr.sh:[0x240001b71:0x3a:0x0], ppos: 0, count: 145387 00000080:00200000:2.0:1787827241.249483:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-flr.sh: write ppos: 0, bytes: 145387 00000080:00200000:2.0:1787827241.249486:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-flr.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.249490:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@0000000003f39d71[0x110, 1, [0x240001b71:0x3a:0x0] hash] 00000020:00200000:2.0:1787827241.249492:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 145387) 0 0 00000080:00200000:2.0:1787827241.249494:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3a:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.249499:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 35] 00020000:00200000:2.0:1787827241.249521:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 145387) 145387, 145387 00020000:00200000:2.0:1787827241.249524:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.249531:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 35] 00000008:00010000:2.0:1787827241.249538:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.249540:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7883461e30, osc lock ffff9a7884063728, flags 0x0 00010000:00010000:2.0:1787827241.249547:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/158 (0 147455) 00010000:00010000:2.0:1787827241.249564:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.249571:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->147455] (req 0->147455) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.249578:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->147455] (req 0->147455) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.249588:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476624128:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.249626:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27000 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.250460:0:47593:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787827241.250470:0:47593:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488782976 00000100:00100000:2.0:1787827241.250474:0:47593:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488782976 00000100:00100000:2.0:1787827241.250501:0:47593:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 269 00000100:00100000:2.0:1787827241.250523:0:47593:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78873da850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:77152:x1874671488782976:12345-192.168.203.140@tcp:104: 00010000:00010000:2.0:1787827241.250531:0:47593:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000369cb29c/0xebdcd48a4298ec7d lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc63a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.250567:0:47593:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787827241.250572:0:47593:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78873da850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:77152:x1874671488782976:12345-192.168.203.140@tcp:104: Request processed in 51us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.250578:0:47593:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 269 00010000:00010000:1.0:1787827241.250595:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.250602:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000369cb29c/0xebdcd48a4298ec7d lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc63a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.250611:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7886bd3200 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.250615:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000369cb29c/0xebdcd48a4298ec7d lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc63a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.250627:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.250633:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.250654:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000369cb29c/0xebdcd48a4298ec7d lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc63a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.250661:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.250678:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000369cb29c/0xebdcd48a4298ec7d lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc63a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.250684:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000369cb29c/0xebdcd48a4298ec7d lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc63a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.250700:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.250703:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.250705:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.251229:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476624128:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.251239:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->147455] (req 0->147455) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251247:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->147455] (req 0->147455) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251252:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->147455] (req 0->147455) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.251516:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488783104 00000100:00100000:2.0:1787827241.251520:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488783104 00000100:00100000:2.0:1787827241.251543:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 270 00000100:00100000:2.0:1787827241.251548:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c9a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488783104:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.251554:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->147455] (req 0->147455) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251561:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->147455] (req 0->147455) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251567:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->147455) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251573:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->147455) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251578:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->147455) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251589:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000002cc7cb05/0xebdcd48a4298ec84 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->147455) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc641 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.251604:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890906400) 00000100:00100000:2.0:1787827241.251640:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c9a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488783104:12345-192.168.203.140@tcp:105: Request processed in 88us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.251656:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 270 00000100:00100000:1.0:1787827241.252187:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476624256:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.257619:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476625152:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.258375:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.258385:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.258393:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.258399:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.258406:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.258412:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.258423:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.258434:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.258439:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.258444:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.258452:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476625152:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.259413:0:47593:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488783232 00000100:00100000:2.0:1787827241.259418:0:47593:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488783232 00000100:00100000:2.0:1787827241.259449:0:47593:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 271 00000100:00100000:2.0:1787827241.259455:0:47593:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:77344:x1874671488783232:12345-192.168.203.140@tcp:104: 00010000:00010000:2.0:1787827241.259462:0:47593:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040840230/0xebdcd48a4298eca0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x9e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc695 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.259496:0:47593:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787827241.259501:0:47593:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:77344:x1874671488783232:12345-192.168.203.140@tcp:104: Request processed in 47us (128us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.259521:0:47593:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 271 00000100:00100000:2.0:1787827241.259651:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476625408:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.260386:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488783360 00000100:00100000:2.0:1787827241.260390:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488783360 00000100:00100000:2.0:1787827241.260415:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 272 00000100:00100000:2.0:1787827241.260420:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838cb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488783360:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.260427:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae2c3401/0xebdcd48a4298eca7 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->172031] (req 0->172031) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc69c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.260434:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae2c3401/0xebdcd48a4298eca7 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->172031] (req 0->172031) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc69c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.260441:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae2c3401/0xebdcd48a4298eca7 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->172031) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc69c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.260446:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae2c3401/0xebdcd48a4298eca7 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->172031) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc69c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.260452:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae2c3401/0xebdcd48a4298eca7 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->172031) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc69c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.260458:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae2c3401/0xebdcd48a4298eca7 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0x9e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->172031) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc69c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.260471:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7886bd3c00) 00000100:00100000:2.0:1787827241.260496:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838cb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488783360:12345-192.168.203.140@tcp:105: Request processed in 74us (144us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.260526:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 272 00000100:00100000:2.0:1787827241.261127:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476625408:192.168.203.140@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787827241.267902:0:10935:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.267912:0:10935:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.267921:0:10935:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.267928:0:10935:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.267936:0:10935:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 5/2,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.267943:0:10935:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.267952:0:10935:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.267958:0:10935:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.267964:0:10935:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.267975:0:10935:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.267992:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476626304:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.268853:0:47593:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488783488 00000100:00100000:2.0:1787827241.268861:0:47593:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488783488 00000100:00100000:2.0:1787827241.268912:0:47593:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 273 00000100:00100000:2.0:1787827241.268924:0:47593:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a665da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:82310:x1874671488783488:12345-192.168.203.140@tcp:104: 00010000:00010000:2.0:1787827241.268935:0:47593:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.268980:0:47593:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787827241.268985:0:47593:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a665da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:82310:x1874671488783488:12345-192.168.203.140@tcp:104: Request processed in 66us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.268992:0:47593:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 273 00010000:00010000:1.0:1787827241.269007:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.269013:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.269020:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a18a00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.269023:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.269034:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.269041:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.269062:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.269068:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.269084:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.269091:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000001619d372/0xebdcd48a4298ecc3 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc6f0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.269102:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.269105:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.269108:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.269638:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65a0b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476626432:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.269650:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.269659:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.269666:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.269978:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488783616 00000100:00100000:2.0:1787827241.269983:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488783616 00000100:00100000:2.0:1787827241.270010:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 274 00000100:00100000:2.0:1787827241.270017:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c9e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488783616:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.270024:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.270032:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->32767] (req 0->32767) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.270042:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.270048:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.270053:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.270060:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000017fff27a/0xebdcd48a4298ecca lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc6f7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.270075:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890a19600) 00000100:00100000:2.0:1787827241.270102:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c9e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488783616:12345-192.168.203.140@tcp:105: Request processed in 85us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.270117:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 274 00000100:00100000:1.0:1787827241.270749:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476626560:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.272643:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838cc3c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476626944:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.272656:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272667:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272675:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272687:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272692:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272696:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a18800 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.272700:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272704:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272708:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272712:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.272715:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272719:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ecd1 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.272734:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78838cc3c0 x1874671476626944/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.272745:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.272757:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.272795:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.272803:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.272810:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.272818:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.272823:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.272826:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.272849:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.272854:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.272857:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.272863:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.272871:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.272875:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.272878:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.272880:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.272882:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.272889:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-lfsck.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:2.0:1787827241.272892:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lfsck.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.272898:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lfsck.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.272915:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272920:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.272929:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.272939:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6671a40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476627072:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.272971:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27240 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.273797:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6671a40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476627072:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.273806:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273813:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273818:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273823:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273827:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273845:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890906600 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.273848:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273853:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273858:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273863:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.273867:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.273872:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ecd8 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.273881:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6671a40 x1874671476627072/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.273888:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.273897:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.273935:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.273942:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.273947:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.273954:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.273959:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.273962:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.273964:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.273967:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:2.0:1787827241.273970:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.273975:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.273982:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.273986:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.273988:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.273990:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.273992:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.273998:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.274001:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=sanity-lfsck.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78894b3200, open_flags 80c1, mode 81ed opened 0 00000080:00200000:2.0:1787827241.274005:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lfsck.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:2.0:1787827241.274014:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lfsck.sh,[0x240001b71:0x3d:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:2.0:1787827241.274033:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.274038:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.274044:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.274048:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6671a40 x1874671476627328/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.274053:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.274062:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6671a40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476627328:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.274129:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27240 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.275319:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6671a40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476627328:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.275329:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275345:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275351:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275358:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275364:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275370:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275374:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275379:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.275386:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6671a40 x1874671476627328/t17179869493(17179869493) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.275394:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6671a40 x1874671476627328/t17179869493(17179869493) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.275402:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.275409:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275415:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/61 (0 0) 00000080:00200000:2.0:1787827241.275418:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.275424:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=sanity-lfsck.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:2.0:1787827241.275439:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x3d:0x0](ffff9a789dfc63c8) 00000080:00200000:2.0:1787827241.275481:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x3d:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.275484:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x3d:0x0](ffff9a789dfc63c8) 00000080:00010000:2.0:1787827241.275487:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3d:0x0](ffff9a789dfc63c8) for lock 0xebdcd48a4298ecdf 00000080:00010000:2.0:1787827241.275491:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3d:0x0](ffff9a789dfc63c8) 00000080:00200008:2.0:1787827241.275496:0:48385:0:(file.c:1090:ll_file_open()) START file sanity-lfsck.sh:[0x240001b71:0x3d:0x0](ffff9a789dfc63c8), flags 100301 00000002:00100000:2.0:1787827241.275527:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6671a40 x1874671476627328/t17179869493(17179869493) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.275551:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ecdf from it ffff9a78877da960 00010000:00010000:2.0:1787827241.275555:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275561:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.275571:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x3d:0x0](ffff9a789dfc63c8), inode mode 81ed mask 40 00000080:00200008:2.0:1787827241.275580:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-lfsck.sh:[0x240001b71:0x3d:0x0](ffff9a789dfc63c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.275583:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x3d:0x0](ffff9a789dfc63c8),name=sanity-lfsck.sh 00000002:00010000:2.0:1787827241.275591:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x3d:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.275597:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.275603:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.275608:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.275611:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3d:0x0](ffff9a789dfc63c8) 00000080:00010000:2.0:1787827241.275613:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3d:0x0](ffff9a789dfc63c8) for lock 0xebdcd48a4298ecdf 00000080:00010000:2.0:1787827241.275616:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ecdf from it ffffaa5487e7bcd8 00010000:00010000:2.0:1787827241.275618:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.275623:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.275632:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3d:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.275636:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x3d:0x0] 00000008:00010000:2.0:1787827241.275644:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.275645:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7883461e30, osc lock ffff9a7884062fd8, flags 0x400001002 00010000:00010000:2.0:1787827241.275676:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.275683:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1787827241.275706:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.275723:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6670000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476627456:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.276491:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.276515:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.276526:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.276533:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.276542:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.276554:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.276561:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.276568:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.276574:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.276580:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.276587:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3d:0x0] updating i_size 0 i_blocks 0 00000100:00100000:1.0:1787827241.276590:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6670000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476627456:192.168.203.140@tcp:101:cp.0 00000020:00010000:2.0:1787827241.276592:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.276596:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.276598:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7883461e30 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.276601:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.276603:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7883461e30 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.276611:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3d:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.276730:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-lfsck.sh:[0x240001b71:0x3d:0x0], ppos: 0, count: 215293 00000080:00200000:2.0:1787827241.276738:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-lfsck.sh: write ppos: 0, bytes: 215293 00000080:00200000:2.0:1787827241.276741:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-lfsck.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.276745:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000ba14c02a[0x110, 1, [0x240001b71:0x3d:0x0] hash] 00000020:00200000:2.0:1787827241.276747:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 215293) 0 0 00000080:00200000:2.0:1787827241.276749:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3d:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.276755:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 52] 00020000:00200000:2.0:1787827241.276760:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 215293) 215293, 215293 00020000:00200000:2.0:1787827241.276763:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.276779:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 52] 00000008:00010000:2.0:1787827241.276785:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.276787:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7883461e30, osc lock ffff9a7884062fd8, flags 0x0 00010000:00010000:2.0:1787827241.276793:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419264/159 (0 217087) 00010000:00010000:2.0:1787827241.276810:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 3/0,1 mode: --/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.276816:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 3/0,1 mode: --/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->217087] (req 0->217087) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.276822:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 3/0,1 mode: --/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->217087] (req 0->217087) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.276858:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6670000 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476627584:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.276900:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.277638:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:1.0:1787827241.277805:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.277812:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.277819:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890907600 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.277824:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.277855:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.277864:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.277897:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.277906:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.277930:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.277940:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000000bf8513e/0xebdcd48a4298ece6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc74b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.277962:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.277965:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.277967:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.278791:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:2.0:1787827241.278956:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.278971:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.278991:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 5/0,2 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.279004:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,2 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.279016:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 5/0,2 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.279023:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.279037:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) sanity-lfsck.sh: write [0, 215293) append 0 append lockpos 0 00000080:00200000:2.0:1787827241.279046:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279076:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279084:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000081cfac78[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 0] 00000080:00200000:2.0:1787827241.279087:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000acb5a2ec 17ffffc0001001 3:0 ffff9a7886c9cae0 0 lru 00000080:00200000:2.0:1787827241.279093:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008d554541 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279095:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000081cfac78 00000080:00200000:2.0:1787827241.279097:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:2.0:1787827241.279099:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279114:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279117:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000079793984[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 1] 00000080:00200000:2.0:1787827241.279120:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003619176e 17ffffc0001001 3:0 ffff9a7886c9c658 1 lru 00000080:00200000:2.0:1787827241.279124:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c16cd5ad 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279125:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000079793984 00000080:00200000:2.0:1787827241.279126:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:2.0:1787827241.279127:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279137:0:48385:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279140:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c33f94a2[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 2] 00000080:00200000:2.0:1787827241.279141:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b4f39272 17ffffc0001001 3:0 ffff9a7886c9c9f8 2 lru 00000080:00200000:2.0:1787827241.279145:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ac0943d3 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279146:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c33f94a2 00000080:00200000:2.0:1787827241.279147:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:2.0:1787827241.279148:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279165:0:48385:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279168:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c1af75d2[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 3] 00000080:00200000:2.0:1787827241.279169:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003b921588 17ffffc0001021 2:0 ffff9a7886c9c910 3 lru 00000080:00200000:2.0:1787827241.279173:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000de33a796 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279175:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c1af75d2 00000080:00200000:2.0:1787827241.279176:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:2.0:1787827241.279177:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279185:0:48385:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279188:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000feab2470[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 4] 00000080:00200000:2.0:1787827241.279190:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e3ecfc31 17ffffc0001001 3:0 ffff9a7886c9cd98 4 lru 00000080:00200000:2.0:1787827241.279193:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003db69394 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279195:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000feab2470 00000080:00200000:2.0:1787827241.279196:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:2.0:1787827241.279197:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279207:0:48385:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279210:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000157284a1[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 5] 00000080:00200000:2.0:1787827241.279212:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000ba8505e 17ffffc0001001 3:0 ffff9a7886c9c3a0 5 lru 00000080:00200000:2.0:1787827241.279216:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001ed9f017 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279218:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000157284a1 00000080:00200000:2.0:1787827241.279219:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:2.0:1787827241.279220:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279238:0:48385:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279241:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000095bd74a9[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 6] 00000080:00200000:2.0:1787827241.279242:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000041d9fe75 17ffffc0001001 3:0 ffff9a7886c9c0e8 6 lru 00000080:00200000:2.0:1787827241.279246:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bdcd106a 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279247:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000095bd74a9 00000080:00200000:2.0:1787827241.279248:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:2.0:1787827241.279250:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279259:0:48385:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279262:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c06b1ab9[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 7] 00000080:00200000:2.0:1787827241.279264:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b71dfd0c 17ffffc0001001 3:0 ffff9a7886c9c2b8 7 lru 00000080:00200000:2.0:1787827241.279268:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d712007a 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279270:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c06b1ab9 00000080:00200000:2.0:1787827241.279270:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:2.0:1787827241.279272:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279280:0:48385:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279283:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d6a788d3[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 8] 00000080:00200000:2.0:1787827241.279285:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cc39af73 17ffffc0001001 3:0 ffff9a7886c9c1d0 8 lru 00000080:00200000:2.0:1787827241.279288:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000009dbfafd 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279290:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d6a788d3 00000080:00200000:2.0:1787827241.279296:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:2.0:1787827241.279297:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279310:0:48385:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279313:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000eca9f10d[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 9] 00000080:00200000:2.0:1787827241.279315:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ccfd4be5 17ffffc0001001 3:0 ffff9a7886c9c740 9 lru 00000080:00200000:2.0:1787827241.279319:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000021fb454c 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279320:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000eca9f10d 00000080:00200000:2.0:1787827241.279321:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:2.0:1787827241.279323:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279332:0:48385:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279335:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000007c8ec7[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 10] 00000080:00200000:2.0:1787827241.279336:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000727006d2 17ffffc0001001 3:0 ffff9a7886c9ce80 10 lru 00000080:00200000:2.0:1787827241.279340:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000810a7419 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279341:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000007c8ec7 00000080:00200000:2.0:1787827241.279342:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:2.0:1787827241.279343:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279352:0:48385:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279355:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e7144267[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 11] 00000080:00200000:2.0:1787827241.279357:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cca79cbd 17ffffc0001001 3:0 ffff9a7886c9c488 11 lru 00000080:00200000:2.0:1787827241.279360:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000749dfa1a 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279362:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e7144267 00000080:00200000:2.0:1787827241.279363:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:2.0:1787827241.279364:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279374:0:48385:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279376:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dee84966[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 12] 00000080:00200000:2.0:1787827241.279378:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000af2a0427 17ffffc0001001 3:0 ffff9a7886c9cbc8 12 lru 00000080:00200000:2.0:1787827241.279382:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c374ed90 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279383:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dee84966 00000080:00200000:2.0:1787827241.279384:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:2.0:1787827241.279385:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279394:0:48385:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279396:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a2ff154f[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 13] 00000080:00200000:2.0:1787827241.279399:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a3a07dc3 17ffffc0001001 3:0 ffff9a7886c9c570 13 lru 00000080:00200000:2.0:1787827241.279402:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cd432090 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279404:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a2ff154f 00000080:00200000:2.0:1787827241.279405:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:2.0:1787827241.279406:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279414:0:48385:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279417:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bc262ef2[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 14] 00000080:00200000:2.0:1787827241.279418:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000b67aa60 17ffffc0001001 3:0 ffff9a7886c9c828 14 lru 00000080:00200000:2.0:1787827241.279423:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a35b5f31 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279424:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bc262ef2 00000080:00200000:2.0:1787827241.279425:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:2.0:1787827241.279426:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279443:0:48385:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279445:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e3b9282a[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 15] 00000080:00200000:2.0:1787827241.279447:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d249a6b1 17ffffc0001001 3:0 ffff9a7890a0bbc8 15 lru 00000080:00200000:2.0:1787827241.279451:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ce350967 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279452:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e3b9282a 00000080:00200000:2.0:1787827241.279453:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:2.0:1787827241.279454:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279464:0:48385:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279469:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d7680a7c[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 16] 00000080:00200000:2.0:1787827241.279472:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d73f6861 17ffffc0001001 3:0 ffff9a7890a0b570 16 lru 00000080:00200000:2.0:1787827241.279478:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000705aff84 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279481:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d7680a7c 00000080:00200000:2.0:1787827241.279483:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000080:00200000:2.0:1787827241.279485:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279516:0:48385:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279523:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000fc71cfd[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 17] 00000080:00200000:2.0:1787827241.279527:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bb277636 17ffffc0001001 3:0 ffff9a7890a0b828 17 lru 00000080:00200000:2.0:1787827241.279534:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e18b43e9 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279538:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000fc71cfd 00000080:00200000:2.0:1787827241.279539:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:2.0:1787827241.279542:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279558:0:48385:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279563:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000823f28ed[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 18] 00000080:00200000:2.0:1787827241.279570:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a41a132d 17ffffc0001021 2:0 ffff9a7890a0b000 18 lru 00000080:00200000:2.0:1787827241.279576:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000062dcddd8 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279578:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000823f28ed 00000080:00200000:2.0:1787827241.279579:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:2.0:1787827241.279581:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279594:0:48385:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279597:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000070504606[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 19] 00000080:00200000:2.0:1787827241.279599:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000076401ef7 17ffffc0001001 3:0 ffff9a7890a0bcb0 19 lru 00000080:00200000:2.0:1787827241.279605:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000512975a7 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279607:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000070504606 00000080:00200000:2.0:1787827241.279608:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:2.0:1787827241.279610:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279625:0:48385:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279630:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007e6a4626[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 20] 00000080:00200000:2.0:1787827241.279633:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000846a9e24 17ffffc0001001 3:0 ffff9a7890a0bae0 20 lru 00000080:00200000:2.0:1787827241.279640:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001d4961d7 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279643:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007e6a4626 00000080:00200000:2.0:1787827241.279644:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:2.0:1787827241.279646:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279660:0:48385:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279664:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000084a091b7[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 21] 00000080:00200000:2.0:1787827241.279667:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d41b04c2 17ffffc0001001 3:0 ffff9a7890a0b658 21 lru 00000080:00200000:2.0:1787827241.279672:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ae0f7f46 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279674:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000084a091b7 00000080:00200000:2.0:1787827241.279675:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:2.0:1787827241.279677:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279687:0:48385:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279689:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a1447ec0[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 22] 00000080:00200000:2.0:1787827241.279692:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000423df289 17ffffc0001001 3:0 ffff9a7890a0b9f8 22 lru 00000080:00200000:2.0:1787827241.279695:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a649e0a3 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279697:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a1447ec0 00000080:00200000:2.0:1787827241.279698:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:2.0:1787827241.279699:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279707:0:48385:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279710:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f2918d42[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 23] 00000080:00200000:2.0:1787827241.279711:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004377e22a 17ffffc0001001 3:0 ffff9a7890a0b910 23 lru 00000080:00200000:2.0:1787827241.279716:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000113ff98b 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279717:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f2918d42 00000080:00200000:2.0:1787827241.279718:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:2.0:1787827241.279719:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279729:0:48385:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279733:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000063dbbc7b[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 24] 00000080:00200000:2.0:1787827241.279736:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b5d3732a 17ffffc0001001 3:0 ffff9a7890a0bd98 24 lru 00000080:00200000:2.0:1787827241.279742:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d77b9323 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279744:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000063dbbc7b 00000080:00200000:2.0:1787827241.279745:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:2.0:1787827241.279747:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279762:0:48385:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279767:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f0bed3f8[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 25] 00000080:00200000:2.0:1787827241.279770:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c1db9018 17ffffc0001001 3:0 ffff9a7890a0b3a0 25 lru 00000080:00200000:2.0:1787827241.279775:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000efe697aa 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279778:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f0bed3f8 00000080:00200000:2.0:1787827241.279779:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:2.0:1787827241.279781:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279798:0:48385:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279802:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a22826a1[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 26] 00000080:00200000:2.0:1787827241.279805:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000030ac3c9b 17ffffc0001001 3:0 ffff9a7890a0b0e8 26 lru 00000080:00200000:2.0:1787827241.279810:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008afb879e 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279813:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a22826a1 00000080:00200000:2.0:1787827241.279814:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:2.0:1787827241.279815:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279846:0:48385:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279853:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c7108654[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 27] 00000080:00200000:2.0:1787827241.279856:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006d345c0b 17ffffc0001001 3:0 ffff9a7890a0b2b8 27 lru 00000080:00200000:2.0:1787827241.279863:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001852dfc2 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279866:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c7108654 00000080:00200000:2.0:1787827241.279868:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:2.0:1787827241.279871:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279887:0:48385:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279892:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000094c2e572[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 28] 00000080:00200000:2.0:1787827241.279895:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009e82a603 17ffffc0001001 3:0 ffff9a7890a0b1d0 28 lru 00000080:00200000:2.0:1787827241.279901:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004c84256b 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279903:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000094c2e572 00000080:00200000:2.0:1787827241.279904:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:2.0:1787827241.279906:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279921:0:48385:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279926:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d2ba9aec[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 29] 00000080:00200000:2.0:1787827241.279929:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000068903417 17ffffc0001001 3:0 ffff9a7890a0b740 29 lru 00000080:00200000:2.0:1787827241.279934:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d1f30a52 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279936:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d2ba9aec 00000080:00200000:2.0:1787827241.279937:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:2.0:1787827241.279939:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279956:0:48385:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279961:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fad82c3b[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 30] 00000080:00200000:2.0:1787827241.279963:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bc6c2271 17ffffc0001001 3:0 ffff9a7890a0be80 30 lru 00000080:00200000:2.0:1787827241.279970:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005f8796da 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.279972:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fad82c3b 00000080:00200000:2.0:1787827241.279973:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:2.0:1787827241.279975:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.279991:0:48385:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:2.0:1787827241.279996:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000093dbf115[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 31] 00000080:00200000:2.0:1787827241.279999:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000eb12ddc7 17ffffc0001001 3:0 ffff9a7890a0b488 31 lru 00000080:00200000:2.0:1787827241.280005:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000082f6e685 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280008:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000093dbf115 00000080:00200000:2.0:1787827241.280009:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:2.0:1787827241.280011:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280033:0:48385:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280038:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d845a632[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 32] 00000080:00200000:2.0:1787827241.280041:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000693a0687 17ffffc0001001 3:0 ffff9a78a640c0e8 32 lru 00000080:00200000:2.0:1787827241.280047:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008d52144d 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280049:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d845a632 00000080:00200000:2.0:1787827241.280050:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:2.0:1787827241.280052:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280072:0:48385:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280077:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fbcc88c4[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 33] 00000080:00200000:2.0:1787827241.280080:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000006bb712f 17ffffc0001021 2:0 ffff9a78a640c2b8 33 lru 00000080:00200000:2.0:1787827241.280085:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001ae797ef 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280087:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fbcc88c4 00000080:00200000:2.0:1787827241.280089:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:2.0:1787827241.280091:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280106:0:48385:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280110:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005b95edf5[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 34] 00000080:00200000:2.0:1787827241.280113:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007db8d45a 17ffffc0001001 3:0 ffff9a78a640c1d0 34 lru 00000080:00200000:2.0:1787827241.280119:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ec8e660a 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280122:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005b95edf5 00000080:00200000:2.0:1787827241.280123:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:2.0:1787827241.280125:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280140:0:48385:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280144:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000047ff9a0f[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 35] 00000080:00200000:2.0:1787827241.280147:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b067b673 17ffffc0001001 3:0 ffff9a78a640c740 35 lru 00000080:00200000:2.0:1787827241.280153:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009a5a3f17 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280155:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000047ff9a0f 00000080:00200000:2.0:1787827241.280156:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:2.0:1787827241.280158:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280173:0:48385:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280177:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f00c516e[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 36] 00000080:00200000:2.0:1787827241.280180:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fb3f9ba1 17ffffc0001001 3:0 ffff9a78a640ce80 36 lru 00000080:00200000:2.0:1787827241.280185:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000db5017f5 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280188:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f00c516e 00000080:00200000:2.0:1787827241.280190:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:2.0:1787827241.280192:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280205:0:48385:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280210:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d7706edd[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 37] 00000080:00200000:2.0:1787827241.280213:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000020bea9bc 17ffffc0001001 3:0 ffff9a78a640c488 37 lru 00000080:00200000:2.0:1787827241.280219:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000dc992b1c 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280221:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d7706edd 00000080:00200000:2.0:1787827241.280223:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:2.0:1787827241.280224:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280239:0:48385:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280244:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000439a703a[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 38] 00000080:00200000:2.0:1787827241.280247:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000511eae8a 17ffffc0001001 3:0 ffff9a78a640cbc8 38 lru 00000080:00200000:2.0:1787827241.280252:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cca159bf 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280255:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000439a703a 00000080:00200000:2.0:1787827241.280256:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:2.0:1787827241.280258:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280273:0:48385:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280277:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000aa951fbb[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 39] 00000080:00200000:2.0:1787827241.280280:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000eef33021 17ffffc0001001 3:0 ffff9a78a640c570 39 lru 00000080:00200000:2.0:1787827241.280286:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c35522de 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280288:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000aa951fbb 00000080:00200000:2.0:1787827241.280290:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:2.0:1787827241.280291:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280309:0:48385:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280313:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008e85c6dc[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 40] 00000080:00200000:2.0:1787827241.280317:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000006a73aa5 17ffffc0001001 3:0 ffff9a78a640c828 40 lru 00000080:00200000:2.0:1787827241.280322:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d85c128f 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280324:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008e85c6dc 00000080:00200000:2.0:1787827241.280325:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:2.0:1787827241.280327:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280343:0:48385:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280347:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000008c5f40d[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 41] 00000080:00200000:2.0:1787827241.280350:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000967f55a7 17ffffc0001001 3:0 ffff9a78a640c000 41 lru 00000080:00200000:2.0:1787827241.280354:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000a672da0 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280356:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000008c5f40d 00000080:00200000:2.0:1787827241.280357:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:2.0:1787827241.280358:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280368:0:48385:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280370:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007238a600[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 42] 00000080:00200000:2.0:1787827241.280373:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b7152f85 17ffffc0001001 3:0 ffff9a78a640ccb0 42 lru 00000080:00200000:2.0:1787827241.280376:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008f37f66d 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280378:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007238a600 00000080:00200000:2.0:1787827241.280379:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:2.0:1787827241.280380:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280388:0:48385:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280391:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005eab1525[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 43] 00000080:00200000:2.0:1787827241.280392:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004c17604b 17ffffc0001001 3:0 ffff9a78a640cae0 43 lru 00000080:00200000:2.0:1787827241.280396:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000489d7736 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280397:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005eab1525 00000080:00200000:2.0:1787827241.280398:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:2.0:1787827241.280400:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280409:0:48385:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280412:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008b6ab2ea[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 44] 00000080:00200000:2.0:1787827241.280414:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000021109cd5 17ffffc0001001 3:0 ffff9a78a640c658 44 lru 00000080:00200000:2.0:1787827241.280418:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f4ec0da5 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280419:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008b6ab2ea 00000080:00200000:2.0:1787827241.280420:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:2.0:1787827241.280421:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280429:0:48385:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280431:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000369e2f71[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 45] 00000080:00200000:2.0:1787827241.280433:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007c89b542 17ffffc0001001 3:0 ffff9a78a640c9f8 45 lru 00000080:00200000:2.0:1787827241.280436:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000020b21e07 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280438:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000369e2f71 00000080:00200000:2.0:1787827241.280439:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:2.0:1787827241.280440:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280452:0:48385:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280457:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002db9eb53[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 46] 00000080:00200000:2.0:1787827241.280460:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001b3fde62 17ffffc0001001 3:0 ffff9a78a640c910 46 lru 00000080:00200000:2.0:1787827241.280466:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000063652e89 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280469:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002db9eb53 00000080:00200000:2.0:1787827241.280471:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:2.0:1787827241.280473:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280489:0:48385:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280494:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008a6e62b9[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 47] 00000080:00200000:2.0:1787827241.280496:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009ac297c9 17ffffc0001001 3:0 ffff9a78a640cd98 47 lru 00000080:00200000:2.0:1787827241.280501:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fe88ab9a 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280521:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008a6e62b9 00000080:00200000:2.0:1787827241.280522:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:2.0:1787827241.280524:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280544:0:48385:0:(rw26.c:875:ll_write_end()) pos 196608, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280549:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000027e61343[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 48] 00000080:00200000:2.0:1787827241.280552:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000892715f8 17ffffc0001021 2:0 ffff9a78a640c3a0 48 lru 00000080:00200000:2.0:1787827241.280557:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009cc3b866 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280560:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000027e61343 00000080:00200000:2.0:1787827241.280561:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:2.0:1787827241.280563:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 49 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280586:0:48385:0:(rw26.c:875:ll_write_end()) pos 200704, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280590:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b4436b28[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 49] 00000080:00200000:2.0:1787827241.280593:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000842b30eb 17ffffc0001001 3:0 ffff9a78a67783a0 49 lru 00000080:00200000:2.0:1787827241.280598:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000dffb452a 49: 1< 0 - - > 2< 200704 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280600:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b4436b28 00000080:00200000:2.0:1787827241.280602:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 50. 00000080:00200000:2.0:1787827241.280603:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 50 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280618:0:48385:0:(rw26.c:875:ll_write_end()) pos 204800, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280622:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002576a79f[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 50] 00000080:00200000:2.0:1787827241.280628:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000059bf3c60 17ffffc0001001 3:0 ffff9a78a67780e8 50 lru 00000080:00200000:2.0:1787827241.280633:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b2770eb8 50: 1< 0 - - > 2< 204800 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280635:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002576a79f 00000080:00200000:2.0:1787827241.280637:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 51. 00000080:00200000:2.0:1787827241.280639:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 51 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.280653:0:48385:0:(rw26.c:875:ll_write_end()) pos 208896, len 4096, copied 4096 00000080:00200000:2.0:1787827241.280657:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000070d955a4[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 51] 00000080:00200000:2.0:1787827241.280661:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000afe6efe3 17ffffc0001001 3:0 ffff9a78a67782b8 51 lru 00000080:00200000:2.0:1787827241.280667:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000077d83e38 51: 1< 0 - - > 2< 208896 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280669:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000070d955a4 00000080:00200000:2.0:1787827241.280670:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 52. 00000080:00200000:2.0:1787827241.280672:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 52 of 0 to 2301 bytes 00000080:00200000:2.0:1787827241.280691:0:48385:0:(rw26.c:875:ll_write_end()) pos 212992, len 2301, copied 2301 00000080:00200000:2.0:1787827241.280694:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bbce0c6e[3 00000000c75ccf1a 2 1 00000000746d8de4 0 0 52] 00000080:00200000:2.0:1787827241.280697:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000064ae41e 17ffffc0001001 3:0 ffff9a78a67781d0 52 lru 00000080:00200000:2.0:1787827241.280703:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c3a88477 52: 1< 0 - - > 2< 212992 0 0 0x0 0x0 | 0000000023a82262 000000008e64ead5 > 3< 0 0 > 4< 0 0 8 3317760 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.280705:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bbce0c6e 00000080:00200000:2.0:1787827241.280706:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 53. 00000080:00200000:2.0:1787827241.280709:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 53, from 0, to 2301 prio 0 00000008:00010000:2.0:1787827241.280733:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280752:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280763:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280773:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280787:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280797:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280807:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280816:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280827:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280870:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280881:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280891:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280902:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280913:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280923:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.280939:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc6578, count 15, dirtied 15 00000008:00010000:2.0:1787827241.280947:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280958:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280969:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280978:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280986:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.280996:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281011:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281021:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281031:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281040:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281050:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281059:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281069:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281078:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281088:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.281102:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc6578, count 15, dirtied 15 00000008:00010000:2.0:1787827241.281108:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281118:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281127:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281137:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281146:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281155:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281165:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281175:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281193:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281202:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281210:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281220:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281230:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281239:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281249:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.281263:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc6578, count 15, dirtied 15 00000008:00010000:2.0:1787827241.281270:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281287:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281297:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281306:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281315:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281325:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281334:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.281342:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000bdf761f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.281351:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc6578, count 8, dirtied 8 00000080:00200000:2.0:1787827241.281354:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 53 pages 215293 bytes, tot: 215293 00000080:00200000:2.0:1787827241.281357:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) sanity-lfsck.sh: write bytes 0, result: 215293 00000020:00010000:2.0:1787827241.281369:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.281371:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.281374:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7883461e30 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:2.0:1787827241.281378:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 5/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.281385:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:2.0:1787827241.281395:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.281396:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7883461e30 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.281401:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 215293 bytes 00000080:00200000:2.0:1787827241.281406:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 52] 00000080:00200000:2.0:1787827241.281411:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3d:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.281414:0:48385:0:(file.c:2240:ll_file_io_generic()) sanity-lfsck.sh: 2 io complete with rc: 0, result: 215293, restart: 0 00000080:00200000:2.0:1787827241.281420:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3d:0x0] updating i_size 215293 i_blocks 0 00000080:00200000:2.0:1787827241.281423:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 215293 00000080:00200008:2.0:1787827241.281444:0:48385:0:(file.c:479:ll_file_release()) START file sanity-lfsck.sh:[0x240001b71:0x3d:0x0](ffff9a789dfc63c8), flags 100001 00000002:00100000:2.0:1787827241.281476:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6671a40 x1874671476627328/t17179869493(17179869493) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787827241.281485:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787827241.281489:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6670000 x1874671476627968/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787827241.281518:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6670000 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476627968:192.168.203.140@tcp:35:cp.0 00000100:00100000:2.0:1787827241.281556:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.282452:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6670000 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476627968:192.168.203.140@tcp:35:cp.0 00000002:00100000:2.0:1787827241.282470:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6671a40 x1874671476627328/t17179869493(17179869493) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.282540:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.282554:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.282568:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.282579:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.282588:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.282593:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.282597:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.282603:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.282606:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.282616:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.282627:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.282633:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.282638:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.282642:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.282646:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.282650:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=sanity-lfsck.sh, flags=8 00000080:00200008:2.0:1787827241.282658:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-lfsck.sh:[0x240001b71:0x3d:0x0](ffff9a789dfc63c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.282663:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x3d:0x0](ffff9a789dfc63c8),name=sanity-lfsck.sh 00000002:00010000:2.0:1787827241.282674:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x3d:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.282684:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.282695:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.282702:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.282707:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3d:0x0](ffff9a789dfc63c8) 00000080:00010000:2.0:1787827241.282710:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3d:0x0](ffff9a789dfc63c8) for lock 0xebdcd48a4298ecdf 00000080:00010000:2.0:1787827241.282714:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ecdf from it ffffaa5487e7bcb8 00010000:00010000:2.0:1787827241.282718:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.282727:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.282743:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3d:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.282751:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x3d:0x0] 00000008:00010000:2.0:1787827241.282761:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.282765:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7883461e30, osc lock ffff9a7884062fd8, flags 0x400001002 00010000:00010000:2.0:1787827241.282776:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.282786:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:2.0:1787827241.282796:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:2.0:1787827241.282800:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 6/0,1 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.282809:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000569432d6/0xebdcd48a4298eced lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x9f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc752 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:2.0:1787827241.282818:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1787827241.282823:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3d:0x0] updating i_size 215293 i_blocks 0 00000020:00010000:2.0:1787827241.282827:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.282852:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.282855:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7883461e30 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.282859:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.282863:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7883461e30 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.282870:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3d:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.282908:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.282916:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.282924:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.282935:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.282942:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.282947:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.282949:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.282954:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.282958:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.282973:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.282982:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.282986:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.282990:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.282993:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.282996:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.283006:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-lnet.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:2.0:1787827241.283010:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lnet.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.283019:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lnet.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.283049:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.283056:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.283065:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.283077:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628096:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.283110:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.284115:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628096:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.284124:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284133:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284141:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284148:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284156:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284163:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890906c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.284168:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284177:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284185:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284192:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.284197:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284202:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906c00/0xebdcd48a4298ecf4 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.284213:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a66730c0 x1874671476628096/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.284222:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.284234:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.284258:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.284264:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.284269:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.284276:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.284281:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.284284:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.284286:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.284289:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.284292:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.284298:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.284306:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.284311:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.284315:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.284318:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.284321:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.284329:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-lnet.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:2.0:1787827241.284333:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lnet.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.284342:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lnet.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.284363:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284372:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.284378:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.284387:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628224:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.284416:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.285375:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628224:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.285383:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285391:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285396:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285401:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285405:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285409:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890907e00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.285411:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285415:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285420:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285424:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.285427:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285431:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907e00/0xebdcd48a4298ecfb lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.285440:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a66730c0 x1874671476628224/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.285448:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.285460:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.285527:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.285535:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.285543:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.285554:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.285562:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.285567:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.285570:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.285575:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:2.0:1787827241.285579:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.285594:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.285604:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.285609:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.285614:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.285617:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.285620:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.285629:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.285635:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=sanity-lnet.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78894b2600, open_flags 80c1, mode 81ed opened 0 00000080:00200000:2.0:1787827241.285642:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lnet.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:2.0:1787827241.285660:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lnet.sh,[0x240001b71:0x3e:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:2.0:1787827241.285691:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.285700:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.285709:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.285716:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66730c0 x1874671476628480/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.285726:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.285739:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628480:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.285856:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.287144:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66730c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628480:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.287168:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287182:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287189:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287198:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287206:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287213:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287218:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287223:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.287230:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a66730c0 x1874671476628480/t17179869495(17179869495) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.287239:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a66730c0 x1874671476628480/t17179869495(17179869495) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.287254:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.287266:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287276:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/62 (0 0) 00000080:00200000:2.0:1787827241.287281:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.287290:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=sanity-lnet.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:2.0:1787827241.287336:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x3e:0x0](ffff9a789ded1148) 00000080:00200000:2.0:1787827241.287418:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x3e:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.287432:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x3e:0x0](ffff9a789ded1148) 00000080:00010000:2.0:1787827241.287434:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3e:0x0](ffff9a789ded1148) for lock 0xebdcd48a4298ed02 00000080:00010000:2.0:1787827241.287442:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3e:0x0](ffff9a789ded1148) 00000080:00200008:2.0:1787827241.287450:0:48385:0:(file.c:1090:ll_file_open()) START file sanity-lnet.sh:[0x240001b71:0x3e:0x0](ffff9a789ded1148), flags 100301 00000002:00100000:2.0:1787827241.287465:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a66730c0 x1874671476628480/t17179869495(17179869495) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.287494:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ed02 from it ffff9a78877da960 00010000:00010000:2.0:1787827241.287498:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287518:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.287532:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x3e:0x0](ffff9a789ded1148), inode mode 81ed mask 40 00000080:00200008:2.0:1787827241.287544:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-lnet.sh:[0x240001b71:0x3e:0x0](ffff9a789ded1148), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.287548:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x3e:0x0](ffff9a789ded1148),name=sanity-lnet.sh 00000002:00010000:2.0:1787827241.287557:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x3e:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.287563:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.287569:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.287574:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.287577:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3e:0x0](ffff9a789ded1148) 00000080:00010000:2.0:1787827241.287579:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3e:0x0](ffff9a789ded1148) for lock 0xebdcd48a4298ed02 00000080:00010000:2.0:1787827241.287581:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ed02 from it ffffaa5487e7bcd8 00010000:00010000:2.0:1787827241.287584:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.287589:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.287599:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3e:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.287606:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x3e:0x0] 00000008:00010000:2.0:1787827241.287617:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.287620:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c3b0, osc lock ffff9a7884062fd8, flags 0x400001002 00010000:00010000:2.0:1787827241.287646:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.287656:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1787827241.287695:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.287713:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476628608:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.288499:0:10934:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.288525:0:10934:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.288538:0:10934:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.288551:0:10934:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.288561:0:10934:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 5/2,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.288569:0:10934:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.288579:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.288585:0:10934:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.288590:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.288596:0:10934:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.288606:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476628608:192.168.203.140@tcp:101:cp.0 00000080:00200000:2.0:1787827241.288607:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3e:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.288613:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.288616:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.288619:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c3b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.288622:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.288624:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c3b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.288633:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3e:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.288754:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-lnet.sh:[0x240001b71:0x3e:0x0], ppos: 0, count: 199288 00000080:00200000:2.0:1787827241.288763:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-lnet.sh: write ppos: 0, bytes: 199288 00000080:00200000:2.0:1787827241.288768:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-lnet.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.288773:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@000000007bd9020d[0x110, 1, [0x240001b71:0x3e:0x0] hash] 00000020:00200000:2.0:1787827241.288775:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 199288) 0 0 00000080:00200000:2.0:1787827241.288777:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3e:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.288784:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 48] 00020000:00200000:2.0:1787827241.288791:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 199288) 199288, 199288 00020000:00200000:2.0:1787827241.288802:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.288814:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 48] 00000008:00010000:2.0:1787827241.288822:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.288824:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c3b0, osc lock ffff9a7884062fd8, flags 0x0 00010000:00010000:2.0:1787827241.288852:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/160 (0 200703) 00010000:00010000:2.0:1787827241.288876:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.288887:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->200703] (req 0->200703) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.288897:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->200703] (req 0->200703) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.288921:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6670f00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628736:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.288959:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27000 going to sleep for 11 seconds 00010000:00010000:1.0:1787827241.289809:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.289817:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.289850:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890906000 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.289855:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.289869:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.289877:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.289901:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.289907:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.289927:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.289937:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ed09 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc7a6 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.289955:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.289959:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.289965:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.290566:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6670f00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476628736:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.290579:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->200703] (req 0->200703) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.290588:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->200703] (req 0->200703) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.290595:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->200703] (req 0->200703) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.290885:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00010000:00010000:2.0:1787827241.291086:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.291095:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.291104:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.291116:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,2 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.291131:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.291137:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.291154:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) sanity-lnet.sh: write [0, 199288) append 0 append lockpos 0 00000080:00200000:2.0:1787827241.291165:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291199:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291208:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000830b60bc[3 0000000076c05945 2 1 00000000746d8de4 0 0 0] 00000080:00200000:2.0:1787827241.291211:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002698dba8 17ffffc0001001 3:0 ffff9a78a6778740 0 lru 00000080:00200000:2.0:1787827241.291218:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c85fc732 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291219:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000830b60bc 00000080:00200000:2.0:1787827241.291221:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:2.0:1787827241.291223:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291241:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291244:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000083f41389[3 0000000076c05945 2 1 00000000746d8de4 0 0 1] 00000080:00200000:2.0:1787827241.291246:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004c281aed 17ffffc0001001 3:0 ffff9a78a6778e80 1 lru 00000080:00200000:2.0:1787827241.291250:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007309e4e8 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291252:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000083f41389 00000080:00200000:2.0:1787827241.291254:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:2.0:1787827241.291255:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291265:0:48385:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291268:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009fa815da[3 0000000076c05945 2 1 00000000746d8de4 0 0 2] 00000080:00200000:2.0:1787827241.291269:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e18f8629 17ffffc0001001 3:0 ffff9a78a6778488 2 lru 00000080:00200000:2.0:1787827241.291273:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cb4b4c3b 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291275:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009fa815da 00000080:00200000:2.0:1787827241.291276:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:2.0:1787827241.291278:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291287:0:48385:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291289:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003aeade4b[3 0000000076c05945 2 1 00000000746d8de4 0 0 3] 00000080:00200000:2.0:1787827241.291292:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000028010104 17ffffc0001001 3:0 ffff9a78a6778bc8 3 lru 00000080:00200000:2.0:1787827241.291298:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000036a38c8f 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291301:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003aeade4b 00000080:00200000:2.0:1787827241.291302:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:2.0:1787827241.291304:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291319:0:48385:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291324:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000703c5ab6[3 0000000076c05945 2 1 00000000746d8de4 0 0 4] 00000080:00200000:2.0:1787827241.291327:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005ec2fd3e 17ffffc0001001 3:0 ffff9a78a6778570 4 lru 00000080:00200000:2.0:1787827241.291334:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000099ff7392 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291336:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000703c5ab6 00000080:00200000:2.0:1787827241.291337:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:2.0:1787827241.291340:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291357:0:48385:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291362:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000045b476e5[3 0000000076c05945 2 1 00000000746d8de4 0 0 5] 00000080:00200000:2.0:1787827241.291365:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000089c4dd92 17ffffc0001001 3:0 ffff9a78a6778828 5 lru 00000080:00200000:2.0:1787827241.291371:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d1ac9568 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291374:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000045b476e5 00000080:00200000:2.0:1787827241.291376:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:2.0:1787827241.291378:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291395:0:48385:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291400:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008d9a37b0[3 0000000076c05945 2 1 00000000746d8de4 0 0 6] 00000080:00200000:2.0:1787827241.291403:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008c502b14 17ffffc0001001 3:0 ffff9a78a6778000 6 lru 00000080:00200000:2.0:1787827241.291408:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000087664fe6 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291409:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008d9a37b0 00000080:00200000:2.0:1787827241.291410:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:2.0:1787827241.291412:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291423:0:48385:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291426:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006de23605[3 0000000076c05945 2 1 00000000746d8de4 0 0 7] 00000080:00200000:2.0:1787827241.291429:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ab9cbd7e 17ffffc0001001 3:0 ffff9a78a6778cb0 7 lru 00000080:00200000:2.0:1787827241.291435:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005a6063e2 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291437:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006de23605 00000080:00200000:2.0:1787827241.291439:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:2.0:1787827241.291440:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291458:0:48385:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291463:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c4d2b941[3 0000000076c05945 2 1 00000000746d8de4 0 0 8] 00000080:00200000:2.0:1787827241.291465:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000221a029c 17ffffc0001001 3:0 ffff9a78a6778ae0 8 lru 00000080:00200000:2.0:1787827241.291470:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fdb6c2f4 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291473:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c4d2b941 00000080:00200000:2.0:1787827241.291475:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:2.0:1787827241.291476:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291492:0:48385:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291497:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000099b5b022[3 0000000076c05945 2 1 00000000746d8de4 0 0 9] 00000080:00200000:2.0:1787827241.291500:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007b5aaaca 17ffffc0001001 3:0 ffff9a78a6778658 9 lru 00000080:00200000:2.0:1787827241.291520:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007f916029 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291523:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000099b5b022 00000080:00200000:2.0:1787827241.291524:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:2.0:1787827241.291527:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291551:0:48385:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291556:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009baa6cee[3 0000000076c05945 2 1 00000000746d8de4 0 0 10] 00000080:00200000:2.0:1787827241.291560:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008a71cecc 17ffffc0001021 2:0 ffff9a78a67789f8 10 lru 00000080:00200000:2.0:1787827241.291566:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b87985a1 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291568:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009baa6cee 00000080:00200000:2.0:1787827241.291569:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:2.0:1787827241.291571:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291586:0:48385:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291589:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b9cac126[3 0000000076c05945 2 1 00000000746d8de4 0 0 11] 00000080:00200000:2.0:1787827241.291591:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000053032fc7 17ffffc0001001 3:0 ffff9a78a6778910 11 lru 00000080:00200000:2.0:1787827241.291595:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fe2b2878 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291597:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b9cac126 00000080:00200000:2.0:1787827241.291598:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:2.0:1787827241.291600:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291608:0:48385:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291612:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001820dd08[3 0000000076c05945 2 1 00000000746d8de4 0 0 12] 00000080:00200000:2.0:1787827241.291614:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000329c48ba 17ffffc0001001 3:0 ffff9a78a6778d98 12 lru 00000080:00200000:2.0:1787827241.291617:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006aa407a2 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291620:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001820dd08 00000080:00200000:2.0:1787827241.291620:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:2.0:1787827241.291621:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291638:0:48385:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291641:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000045bb78d0[3 0000000076c05945 2 1 00000000746d8de4 0 0 13] 00000080:00200000:2.0:1787827241.291643:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b089fb09 17ffffc0001001 3:0 ffff9a78887cacb0 13 lru 00000080:00200000:2.0:1787827241.291647:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003aba6b02 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291648:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000045bb78d0 00000080:00200000:2.0:1787827241.291650:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:2.0:1787827241.291651:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291661:0:48385:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291664:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009bd72fb5[3 0000000076c05945 2 1 00000000746d8de4 0 0 14] 00000080:00200000:2.0:1787827241.291666:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006b877451 17ffffc0001001 3:0 ffff9a78887caae0 14 lru 00000080:00200000:2.0:1787827241.291671:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a40b345b 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291673:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009bd72fb5 00000080:00200000:2.0:1787827241.291674:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:2.0:1787827241.291676:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291689:0:48385:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291692:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000067128519[3 0000000076c05945 2 1 00000000746d8de4 0 0 15] 00000080:00200000:2.0:1787827241.291694:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e99d8b5c 17ffffc0001001 3:0 ffff9a78887ca658 15 lru 00000080:00200000:2.0:1787827241.291698:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ec57c8d7 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291700:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000067128519 00000080:00200000:2.0:1787827241.291701:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:2.0:1787827241.291703:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291713:0:48385:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291716:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f2da0607[3 0000000076c05945 2 1 00000000746d8de4 0 0 16] 00000080:00200000:2.0:1787827241.291718:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000f569d3c 17ffffc0001001 3:0 ffff9a78887ca9f8 16 lru 00000080:00200000:2.0:1787827241.291723:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d89b0f08 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291724:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f2da0607 00000080:00200000:2.0:1787827241.291725:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000080:00200000:2.0:1787827241.291726:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291736:0:48385:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291743:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ddf01580[3 0000000076c05945 2 1 00000000746d8de4 0 0 17] 00000080:00200000:2.0:1787827241.291747:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000f61e6ba 17ffffc0001001 3:0 ffff9a78887ca910 17 lru 00000080:00200000:2.0:1787827241.291751:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002895abdc 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291752:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ddf01580 00000080:00200000:2.0:1787827241.291753:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:2.0:1787827241.291754:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291764:0:48385:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291773:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000c9e6b93[3 0000000076c05945 2 1 00000000746d8de4 0 0 18] 00000080:00200000:2.0:1787827241.291775:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c10e00a8 17ffffc0001001 3:0 ffff9a78887cad98 18 lru 00000080:00200000:2.0:1787827241.291779:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ba0ac974 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291780:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000c9e6b93 00000080:00200000:2.0:1787827241.291781:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:2.0:1787827241.291783:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291793:0:48385:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291795:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004344a1a8[3 0000000076c05945 2 1 00000000746d8de4 0 0 19] 00000080:00200000:2.0:1787827241.291797:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000098196adb 17ffffc0001001 3:0 ffff9a78887ca3a0 19 lru 00000080:00200000:2.0:1787827241.291801:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002627ff87 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291803:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004344a1a8 00000080:00200000:2.0:1787827241.291803:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:2.0:1787827241.291804:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291813:0:48385:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291815:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000075c78416[3 0000000076c05945 2 1 00000000746d8de4 0 0 20] 00000080:00200000:2.0:1787827241.291817:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000011f25b8c 17ffffc0001001 3:0 ffff9a78887ca0e8 20 lru 00000080:00200000:2.0:1787827241.291820:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000374c7ec3 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291822:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000075c78416 00000080:00200000:2.0:1787827241.291823:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:2.0:1787827241.291824:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291851:0:48385:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291855:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c5d500d2[3 0000000076c05945 2 1 00000000746d8de4 0 0 21] 00000080:00200000:2.0:1787827241.291858:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ae21a2b7 17ffffc0001001 3:0 ffff9a78887ca2b8 21 lru 00000080:00200000:2.0:1787827241.291862:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006dcdb3b4 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291864:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c5d500d2 00000080:00200000:2.0:1787827241.291865:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:2.0:1787827241.291866:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291877:0:48385:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291880:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000040406f7f[3 0000000076c05945 2 1 00000000746d8de4 0 0 22] 00000080:00200000:2.0:1787827241.291881:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ffb88c3f 17ffffc0001001 3:0 ffff9a78887ca1d0 22 lru 00000080:00200000:2.0:1787827241.291885:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000083b86461 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291887:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000040406f7f 00000080:00200000:2.0:1787827241.291887:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:2.0:1787827241.291888:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291897:0:48385:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291899:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000064bfa34b[3 0000000076c05945 2 1 00000000746d8de4 0 0 23] 00000080:00200000:2.0:1787827241.291901:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f4875eb9 17ffffc0001001 3:0 ffff9a78887ca740 23 lru 00000080:00200000:2.0:1787827241.291904:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cebf8b2f 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291906:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000064bfa34b 00000080:00200000:2.0:1787827241.291907:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:2.0:1787827241.291908:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291917:0:48385:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291919:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000075ed8a35[3 0000000076c05945 2 1 00000000746d8de4 0 0 24] 00000080:00200000:2.0:1787827241.291922:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001a051c72 17ffffc0001001 3:0 ffff9a78887cae80 24 lru 00000080:00200000:2.0:1787827241.291925:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a47e1172 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291927:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000075ed8a35 00000080:00200000:2.0:1787827241.291928:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:2.0:1787827241.291929:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291940:0:48385:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291942:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c4d6435c[3 0000000076c05945 2 1 00000000746d8de4 0 0 25] 00000080:00200000:2.0:1787827241.291944:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b976c539 17ffffc0001021 2:0 ffff9a78887ca488 25 lru 00000080:00200000:2.0:1787827241.291948:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000aa8a92e7 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291950:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c4d6435c 00000080:00200000:2.0:1787827241.291951:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:2.0:1787827241.291952:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291960:0:48385:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291962:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000739dba88[3 0000000076c05945 2 1 00000000746d8de4 0 0 26] 00000080:00200000:2.0:1787827241.291964:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000048a397bc 17ffffc0001001 3:0 ffff9a78887cabc8 26 lru 00000080:00200000:2.0:1787827241.291967:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008c29695b 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291969:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000739dba88 00000080:00200000:2.0:1787827241.291970:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:2.0:1787827241.291971:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291980:0:48385:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:2.0:1787827241.291982:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b41021cd[3 0000000076c05945 2 1 00000000746d8de4 0 0 27] 00000080:00200000:2.0:1787827241.291984:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004e93a3b6 17ffffc0001001 3:0 ffff9a78887ca570 27 lru 00000080:00200000:2.0:1787827241.291988:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000088303154 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.291989:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b41021cd 00000080:00200000:2.0:1787827241.291990:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:2.0:1787827241.291991:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.291999:0:48385:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292002:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e2e5c528[3 0000000076c05945 2 1 00000000746d8de4 0 0 28] 00000080:00200000:2.0:1787827241.292004:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000047921e02 17ffffc0001001 3:0 ffff9a78887ca828 28 lru 00000080:00200000:2.0:1787827241.292007:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001736f792 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292008:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e2e5c528 00000080:00200000:2.0:1787827241.292009:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:2.0:1787827241.292010:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292022:0:48385:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292025:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c5259cd5[3 0000000076c05945 2 1 00000000746d8de4 0 0 29] 00000080:00200000:2.0:1787827241.292026:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006ccaf0f5 17ffffc0001001 3:0 ffff9a78887ca000 29 lru 00000080:00200000:2.0:1787827241.292030:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a542b1b3 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292031:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c5259cd5 00000080:00200000:2.0:1787827241.292033:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:2.0:1787827241.292034:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292048:0:48385:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292051:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000af5b4d8e[3 0000000076c05945 2 1 00000000746d8de4 0 0 30] 00000080:00200000:2.0:1787827241.292053:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b104dd68 17ffffc0001001 3:0 ffff9a788857e3a0 30 lru 00000080:00200000:2.0:1787827241.292057:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f65235d4 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292059:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000af5b4d8e 00000080:00200000:2.0:1787827241.292059:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:2.0:1787827241.292060:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292069:0:48385:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292071:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000633b438b[3 0000000076c05945 2 1 00000000746d8de4 0 0 31] 00000080:00200000:2.0:1787827241.292073:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000037876fe5 17ffffc0001001 3:0 ffff9a788857e0e8 31 lru 00000080:00200000:2.0:1787827241.292076:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000db94c651 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292078:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000633b438b 00000080:00200000:2.0:1787827241.292079:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:2.0:1787827241.292080:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292088:0:48385:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292091:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006c43760d[3 0000000076c05945 2 1 00000000746d8de4 0 0 32] 00000080:00200000:2.0:1787827241.292093:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003004e2b6 17ffffc0001001 3:0 ffff9a788857e2b8 32 lru 00000080:00200000:2.0:1787827241.292097:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006b4a469c 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292098:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006c43760d 00000080:00200000:2.0:1787827241.292099:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:2.0:1787827241.292100:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292108:0:48385:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292110:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004fa654c7[3 0000000076c05945 2 1 00000000746d8de4 0 0 33] 00000080:00200000:2.0:1787827241.292112:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000666d02ed 17ffffc0001001 3:0 ffff9a788857e1d0 33 lru 00000080:00200000:2.0:1787827241.292115:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006fcfc3e1 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292117:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004fa654c7 00000080:00200000:2.0:1787827241.292118:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:2.0:1787827241.292119:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292127:0:48385:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292130:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ae0a0484[3 0000000076c05945 2 1 00000000746d8de4 0 0 34] 00000080:00200000:2.0:1787827241.292132:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f348c4ac 17ffffc0001001 3:0 ffff9a788857e740 34 lru 00000080:00200000:2.0:1787827241.292136:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000087ae48ea 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292138:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ae0a0484 00000080:00200000:2.0:1787827241.292138:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:2.0:1787827241.292140:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292148:0:48385:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292151:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a88faa58[3 0000000076c05945 2 1 00000000746d8de4 0 0 35] 00000080:00200000:2.0:1787827241.292152:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c1667514 17ffffc0001001 3:0 ffff9a788857ee80 35 lru 00000080:00200000:2.0:1787827241.292156:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a0863a3a 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292157:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a88faa58 00000080:00200000:2.0:1787827241.292158:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:2.0:1787827241.292159:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292168:0:48385:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292171:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ca23aa77[3 0000000076c05945 2 1 00000000746d8de4 0 0 36] 00000080:00200000:2.0:1787827241.292173:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009dd79016 17ffffc0001001 3:0 ffff9a788857e488 36 lru 00000080:00200000:2.0:1787827241.292177:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f7085cc5 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292178:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ca23aa77 00000080:00200000:2.0:1787827241.292179:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:2.0:1787827241.292180:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292188:0:48385:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292191:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e2de1f23[3 0000000076c05945 2 1 00000000746d8de4 0 0 37] 00000080:00200000:2.0:1787827241.292192:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003b4b0168 17ffffc0001001 3:0 ffff9a788857ebc8 37 lru 00000080:00200000:2.0:1787827241.292196:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002d986f95 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292197:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e2de1f23 00000080:00200000:2.0:1787827241.292198:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:2.0:1787827241.292199:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292206:0:48385:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292209:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e3a6e4fe[3 0000000076c05945 2 1 00000000746d8de4 0 0 38] 00000080:00200000:2.0:1787827241.292210:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000bc88fb1 17ffffc0001001 3:0 ffff9a788857e570 38 lru 00000080:00200000:2.0:1787827241.292214:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002a351c09 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292215:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e3a6e4fe 00000080:00200000:2.0:1787827241.292216:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:2.0:1787827241.292217:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292226:0:48385:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292228:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cc13f55a[3 0000000076c05945 2 1 00000000746d8de4 0 0 39] 00000080:00200000:2.0:1787827241.292230:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000647aab9c 17ffffc0001001 3:0 ffff9a788857e828 39 lru 00000080:00200000:2.0:1787827241.292234:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d8de6c3f 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292235:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cc13f55a 00000080:00200000:2.0:1787827241.292235:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:2.0:1787827241.292237:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292247:0:48385:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292250:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b388abe9[3 0000000076c05945 2 1 00000000746d8de4 0 0 40] 00000080:00200000:2.0:1787827241.292252:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000036afaa49 17ffffc0001021 2:0 ffff9a788857e000 40 lru 00000080:00200000:2.0:1787827241.292256:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fd8fff14 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292257:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b388abe9 00000080:00200000:2.0:1787827241.292258:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:2.0:1787827241.292259:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292267:0:48385:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292270:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009cde470f[3 0000000076c05945 2 1 00000000746d8de4 0 0 41] 00000080:00200000:2.0:1787827241.292271:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003d6974c8 17ffffc0001001 3:0 ffff9a788857ecb0 41 lru 00000080:00200000:2.0:1787827241.292275:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cb66a0fe 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292276:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009cde470f 00000080:00200000:2.0:1787827241.292277:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:2.0:1787827241.292278:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292287:0:48385:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292289:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000beb73626[3 0000000076c05945 2 1 00000000746d8de4 0 0 42] 00000080:00200000:2.0:1787827241.292291:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000012d57987 17ffffc0001001 3:0 ffff9a788857eae0 42 lru 00000080:00200000:2.0:1787827241.292295:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b7619917 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292297:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000beb73626 00000080:00200000:2.0:1787827241.292297:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:2.0:1787827241.292298:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292308:0:48385:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292310:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000843048d4[3 0000000076c05945 2 1 00000000746d8de4 0 0 43] 00000080:00200000:2.0:1787827241.292312:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a3269f69 17ffffc0001001 3:0 ffff9a788857e658 43 lru 00000080:00200000:2.0:1787827241.292316:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004f209969 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292317:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000843048d4 00000080:00200000:2.0:1787827241.292318:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:2.0:1787827241.292319:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292327:0:48385:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292329:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005342137e[3 0000000076c05945 2 1 00000000746d8de4 0 0 44] 00000080:00200000:2.0:1787827241.292331:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008ddd6901 17ffffc0001001 3:0 ffff9a788857e9f8 44 lru 00000080:00200000:2.0:1787827241.292334:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000016f3bdcb 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292336:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005342137e 00000080:00200000:2.0:1787827241.292336:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:2.0:1787827241.292337:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292345:0:48385:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292348:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e76c3efe[3 0000000076c05945 2 1 00000000746d8de4 0 0 45] 00000080:00200000:2.0:1787827241.292350:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000004e20c76 17ffffc0001001 3:0 ffff9a788857e910 45 lru 00000080:00200000:2.0:1787827241.292353:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003141c4a1 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292355:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e76c3efe 00000080:00200000:2.0:1787827241.292356:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:2.0:1787827241.292357:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292365:0:48385:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292368:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000051f8ef16[3 0000000076c05945 2 1 00000000746d8de4 0 0 46] 00000080:00200000:2.0:1787827241.292370:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e7cbe5d3 17ffffc0001001 3:0 ffff9a788857ed98 46 lru 00000080:00200000:2.0:1787827241.292373:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000aa0d6c4 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292374:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000051f8ef16 00000080:00200000:2.0:1787827241.292375:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:2.0:1787827241.292376:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.292391:0:48385:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:2.0:1787827241.292393:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f4777479[3 0000000076c05945 2 1 00000000746d8de4 0 0 47] 00000080:00200000:2.0:1787827241.292395:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000de89ba2b 17ffffc0001001 3:0 ffff9a78a65219f8 47 lru 00000080:00200000:2.0:1787827241.292398:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000aa66755e 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292399:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f4777479 00000080:00200000:2.0:1787827241.292401:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:2.0:1787827241.292402:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 2680 bytes 00000080:00200000:2.0:1787827241.292415:0:48385:0:(rw26.c:875:ll_write_end()) pos 196608, len 2680, copied 2680 00000080:00200000:2.0:1787827241.292418:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000810bd55b[3 0000000076c05945 2 1 00000000746d8de4 0 0 48] 00000080:00200000:2.0:1787827241.292420:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f586edda 17ffffc0001001 3:0 ffff9a78a6521910 48 lru 00000080:00200000:2.0:1787827241.292424:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000c2185f2 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000043fddcb7 00000000cc3a5827 > 3< 0 0 > 4< 0 0 8 2428928 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.292425:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000810bd55b 00000080:00200000:2.0:1787827241.292426:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:2.0:1787827241.292428:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 49, from 0, to 2680 prio 0 00000008:00010000:2.0:1787827241.292449:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292465:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292475:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292486:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292495:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292518:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292529:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292543:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292549:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292554:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292559:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292564:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292568:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292573:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292578:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.292589:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded12f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.292593:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292600:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292605:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292627:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292633:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292639:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292645:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292650:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292655:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292664:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292670:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292675:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292679:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292684:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292689:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.292696:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded12f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.292700:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292705:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292715:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292720:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292725:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292730:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292735:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292740:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292745:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292749:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292754:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292761:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292766:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292771:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292776:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.292783:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded12f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.292786:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292791:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292796:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.292801:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000007b3fe7ad ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.292809:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded12f8, count 4, dirtied 4 00000080:00200000:2.0:1787827241.292812:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 49 pages 199288 bytes, tot: 199288 00000080:00200000:2.0:1787827241.292814:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) sanity-lnet.sh: write bytes 0, result: 199288 00000020:00010000:2.0:1787827241.292848:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.292852:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.292855:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c3b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:2.0:1787827241.292861:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.292871:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:2.0:1787827241.292884:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.292888:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c3b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.292898:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 199288 bytes 00000080:00200000:2.0:1787827241.292905:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 48] 00000080:00200000:2.0:1787827241.292913:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3e:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.292917:0:48385:0:(file.c:2240:ll_file_io_generic()) sanity-lnet.sh: 2 io complete with rc: 0, result: 199288, restart: 0 00000080:00200000:2.0:1787827241.292925:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3e:0x0] updating i_size 199288 i_blocks 0 00000080:00200000:2.0:1787827241.292929:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 199288 00000080:00200008:2.0:1787827241.292951:0:48385:0:(file.c:479:ll_file_release()) START file sanity-lnet.sh:[0x240001b71:0x3e:0x0](ffff9a789ded1148), flags 100001 00000002:00100000:2.0:1787827241.292981:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a66730c0 x1874671476628480/t17179869495(17179869495) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787827241.292990:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787827241.292994:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6670f00 x1874671476629120/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787827241.293009:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6670f00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629120:192.168.203.140@tcp:35:cp.0 00000100:00100000:2.0:1787827241.293040:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27000 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.293806:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6670f00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629120:192.168.203.140@tcp:35:cp.0 00000002:00100000:2.0:1787827241.293821:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a66730c0 x1874671476628480/t17179869495(17179869495) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.293881:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.293892:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.293904:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.293912:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.293918:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.293922:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.293924:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.293929:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.293931:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.293937:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.293944:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.293949:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.293953:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.293955:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.293958:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.293960:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=sanity-lnet.sh, flags=8 00000080:00200008:2.0:1787827241.293966:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-lnet.sh:[0x240001b71:0x3e:0x0](ffff9a789ded1148), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.293969:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x3e:0x0](ffff9a789ded1148),name=sanity-lnet.sh 00000002:00010000:2.0:1787827241.293976:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x3e:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.293982:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.293988:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.293992:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.293995:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3e:0x0](ffff9a789ded1148) 00000080:00010000:2.0:1787827241.293996:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3e:0x0](ffff9a789ded1148) for lock 0xebdcd48a4298ed02 00000080:00010000:2.0:1787827241.293998:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ed02 from it ffffaa5487e7bcb8 00010000:00010000:2.0:1787827241.294000:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.294006:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.294017:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3e:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.294023:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x3e:0x0] 00000008:00010000:2.0:1787827241.294029:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.294032:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c3b0, osc lock ffff9a7884062fd8, flags 0x400001002 00010000:00010000:2.0:1787827241.294039:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.294051:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:2.0:1787827241.294057:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:2.0:1787827241.294059:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.294067:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000935cf032/0xebdcd48a4298ed10 lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->200703) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dc7ad expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:2.0:1787827241.294077:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1787827241.294083:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3e:0x0] updating i_size 199288 i_blocks 0 00000020:00010000:2.0:1787827241.294087:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.294090:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.294094:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c3b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.294097:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.294100:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c3b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.294107:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3e:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.294151:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.294160:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.294167:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.294176:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.294189:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.294193:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.294196:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.294200:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.294203:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.294211:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.294220:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.294224:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.294228:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.294231:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.294234:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.294244:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-lsnapshot.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:2.0:1787827241.294248:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lsnapshot.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.294258:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lsnapshot.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.294289:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.294298:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.294307:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.294322:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6671e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629248:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.294362:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27000 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.295443:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6671e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629248:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.295452:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295460:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295466:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295471:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295477:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295482:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890907c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.295486:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295492:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295497:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295515:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.295521:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295530:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907c00/0xebdcd48a4298ed17 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.295542:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6671e00 x1874671476629248/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.295550:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.295563:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.295590:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.295597:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.295603:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.295610:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.295615:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.295618:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.295620:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.295624:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.295626:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.295632:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.295639:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.295642:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.295645:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.295647:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.295649:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.295655:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-lsnapshot.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:2.0:1787827241.295657:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lsnapshot.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.295662:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lsnapshot.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.295677:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295682:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.295688:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.295696:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6671e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629376:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.295726:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27000 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.296696:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6671e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629376:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.296704:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296710:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296714:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296718:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296726:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296730:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890906e00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.296732:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296736:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296740:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296743:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.296746:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296750:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298ed1e lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.296757:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6671e00 x1874671476629376/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.296765:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.296773:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.296813:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.296819:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.296823:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.296852:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.296857:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.296861:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.296863:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.296867:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:2.0:1787827241.296870:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.296876:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.296882:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.296885:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.296888:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.296890:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.296892:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.296900:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.296904:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=sanity-lsnapshot.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78894b3980, open_flags 80c1, mode 81ed opened 0 00000080:00200000:2.0:1787827241.296912:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-lsnapshot.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:2.0:1787827241.296929:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-lsnapshot.sh,[0x240001b71:0x3f:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:2.0:1787827241.296961:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.296968:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.296978:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.296984:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6671e00 x1874671476629632/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.296990:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.297000:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6671e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629632:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.297094:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27000 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.298443:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6671e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629632:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.298454:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298468:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298475:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298483:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298491:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298497:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298525:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298530:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.298538:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6671e00 x1874671476629632/t17179869497(17179869497) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.298548:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6671e00 x1874671476629632/t17179869497(17179869497) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.298562:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.298575:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298585:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/63 (0 0) 00000080:00200000:2.0:1787827241.298590:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.298599:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=sanity-lsnapshot.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:2.0:1787827241.298625:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x3f:0x0](ffff9a789ded00c8) 00000080:00200000:2.0:1787827241.298693:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x3f:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.298696:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x3f:0x0](ffff9a789ded00c8) 00000080:00010000:2.0:1787827241.298699:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3f:0x0](ffff9a789ded00c8) for lock 0xebdcd48a4298ed25 00000080:00010000:2.0:1787827241.298705:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3f:0x0](ffff9a789ded00c8) 00000080:00200008:2.0:1787827241.298714:0:48385:0:(file.c:1090:ll_file_open()) START file sanity-lsnapshot.sh:[0x240001b71:0x3f:0x0](ffff9a789ded00c8), flags 100301 00000002:00100000:2.0:1787827241.298729:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6671e00 x1874671476629632/t17179869497(17179869497) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.298758:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ed25 from it ffff9a78877da960 00010000:00010000:2.0:1787827241.298762:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298768:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.298780:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x3f:0x0](ffff9a789ded00c8), inode mode 81ed mask 40 00000080:00200008:2.0:1787827241.298791:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-lsnapshot.sh:[0x240001b71:0x3f:0x0](ffff9a789ded00c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.298794:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x3f:0x0](ffff9a789ded00c8),name=sanity-lsnapshot.sh 00000002:00010000:2.0:1787827241.298804:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x3f:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.298810:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.298817:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.298822:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.298825:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x3f:0x0](ffff9a789ded00c8) 00000080:00010000:2.0:1787827241.298826:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x3f:0x0](ffff9a789ded00c8) for lock 0xebdcd48a4298ed25 00000080:00010000:2.0:1787827241.298847:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ed25 from it ffffaa5487e7bcd8 00010000:00010000:2.0:1787827241.298849:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.298856:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.298870:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3f:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.298877:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x3f:0x0] 00000008:00010000:2.0:1787827241.298889:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.298891:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c7b0, osc lock ffff9a7884062fd8, flags 0x400001002 00010000:00010000:2.0:1787827241.298916:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.298923:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1787827241.298959:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1787827241.299944:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x3f:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.299951:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.299956:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.299960:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c7b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.299964:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.299967:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c7b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.299979:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3f:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.300019:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-lsnapshot.sh:[0x240001b71:0x3f:0x0], ppos: 0, count: 10934 00000080:00200000:2.0:1787827241.300029:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-lsnapshot.sh: write ppos: 0, bytes: 10934 00000080:00200000:2.0:1787827241.300035:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-lsnapshot.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.300042:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000d3019a0e[0x110, 1, [0x240001b71:0x3f:0x0] hash] 00000020:00200000:2.0:1787827241.300045:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 10934) 0 0 00000080:00200000:2.0:1787827241.300048:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3f:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.300058:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 2] 00020000:00200000:2.0:1787827241.300067:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 10934) 10934, 10934 00020000:00200000:2.0:1787827241.300071:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.300086:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 2] 00000008:00010000:2.0:1787827241.300096:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.300100:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c7b0, osc lock ffff9a7884062fd8, flags 0x0 00010000:00010000:2.0:1787827241.300109:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419264/160 (0 12287) 00010000:00010000:2.0:1787827241.300129:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.300140:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.300150:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.300167:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629888:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.300211:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.301035:0:47593:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488784256 00000100:00100000:2.0:1787827241.301041:0:47593:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488784256 00000100:00100000:2.0:1787827241.301081:0:47593:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 279 00000100:00100000:2.0:1787827241.301090:0:47593:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838cc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:82310:x1874671488784256:12345-192.168.203.140@tcp:104: 00010000:00010000:2.0:1787827241.301100:0:47593:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc801 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.301148:0:47593:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787827241.301156:0:47593:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838cc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:82310:x1874671488784256:12345-192.168.203.140@tcp:104: Request processed in 66us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.301165:0:47593:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 279 00010000:00010000:1.0:1787827241.301179:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787827241.301188:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc801 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.301197:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890907e00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:1.0:1787827241.301202:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc801 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827241.301216:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787827241.301223:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1787827241.301247:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc801 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.301254:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787827241.301277:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc801 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.301284:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ed2c lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc801 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.301295:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.301298:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.301307:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.301823:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476629888:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.301856:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.301870:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.301881:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.302212:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488784384 00000100:00100000:2.0:1787827241.302217:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488784384 00000100:00100000:2.0:1787827241.302252:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 280 00000100:00100000:2.0:1787827241.302261:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a665da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488784384:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.302271:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.302282:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->12287] (req 0->12287) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.302293:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.302304:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.302315:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.302330:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000ae0e5bb7/0xebdcd48a4298ed33 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc808 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.302348:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890906c00) 00000100:00100000:2.0:1787827241.302376:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a665da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488784384:12345-192.168.203.140@tcp:105: Request processed in 115us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.302390:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 280 00000100:00100000:2.0:1787827241.302958:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476630016:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.308789:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476630912:192.168.203.140@tcp:101:cp.0 00000080:00200000:2.0:1787827241.309682:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x40:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.309690:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.309694:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.309703:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a789081d630 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.309707:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.309709:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a789081d630 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.309718:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x40:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.309880:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanityn.sh:[0x240001b71:0x40:0x0], ppos: 0, count: 227516 00000080:00200000:2.0:1787827241.309891:0:48385:0:(file.c:2090:ll_file_io_generic()) sanityn.sh: write ppos: 0, bytes: 227516 00000080:00200000:2.0:1787827241.309896:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanityn.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.309908:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@0000000016a788f0[0x110, 1, [0x240001b71:0x40:0x0] hash] 00000020:00200000:2.0:1787827241.309910:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 227516) 0 0 00000080:00200000:2.0:1787827241.309913:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x40:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.309921:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 55] 00020000:00200000:2.0:1787827241.309928:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 227516) 227516, 227516 00020000:00200000:2.0:1787827241.309932:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.309943:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 55] 00000008:00010000:2.0:1787827241.309951:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.309955:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c2b0, osc lock ffff9a7884062af8, flags 0x0 00010000:00010000:2.0:1787827241.309966:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/161 (0 229375) 00010000:00010000:2.0:1787827241.309988:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.309999:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->229375] (req 0->229375) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.310010:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->229375] (req 0->229375) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.310027:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476631040:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.310059:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.310774:0:47593:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787827241.310790:0:47593:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488784512 00000100:00100000:2.0:1787827241.310796:0:47593:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488784512 00000100:00100000:1.0:1787827241.310896:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 281 00000100:00100000:1.0:1787827241.310908:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a645d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488784512:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.310917:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ed4f lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc85c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.310961:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.310966:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a645d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488784512:12345-192.168.203.140@tcp:104: Request processed in 63us (244us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.310977:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 281 00000100:00100000:1.0:1787827241.311156:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476631168:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.311683:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476631040:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.311695:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->229375] (req 0->229375) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.311703:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->229375] (req 0->229375) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.311711:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->229375] (req 0->229375) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.311971:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488784640 00000100:00100000:2.0:1787827241.311975:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488784640 00000100:00100000:2.0:1787827241.312002:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 282 00000100:00100000:2.0:1787827241.312008:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488784640:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.312015:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->229375] (req 0->229375) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.312024:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->229375] (req 0->229375) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.312031:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.312038:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.312043:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.312049:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000061107bd4/0xebdcd48a4298ed56 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc863 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.312063:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890906800) 00000100:00100000:2.0:1787827241.312094:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488784640:12345-192.168.203.140@tcp:105: Request processed in 83us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.312109:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 282 00000100:00100000:1.0:1787827241.312689:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476631168:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.322767:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488784768 00000100:00100000:1.0:1787827241.322775:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488784768 00000100:00100000:1.0:1787827241.322862:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 283 00000100:00100000:1.0:1787827241.322879:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488784768:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.322895:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040b36cc6/0xebdcd48a4298ed72 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc8b7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.322964:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.322973:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488784768:12345-192.168.203.140@tcp:104: Request processed in 99us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.322984:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 283 00010000:00010000:2.0:1787827241.323012:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827241.323024:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040b36cc6/0xebdcd48a4298ed72 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc8b7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.323046:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a18600 already unused, calling callback (ffffffffc125b410) 00010000:00010000:2.0:1787827241.323054:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040b36cc6/0xebdcd48a4298ed72 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc8b7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.323073:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x41:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.323083:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x41:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827241.323120:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040b36cc6/0xebdcd48a4298ed72 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc8b7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.323131:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.323159:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040b36cc6/0xebdcd48a4298ed72 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc8b7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.323169:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040b36cc6/0xebdcd48a4298ed72 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc8b7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.323188:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.323192:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.323198:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.323731:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667d680 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476632192:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.323749:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->159743] (req 0->159743) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.323762:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->159743] (req 0->159743) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.323773:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->159743] (req 0->159743) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.324144:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488784896 00000100:00100000:2.0:1787827241.324148:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488784896 00000100:00100000:2.0:1787827241.324178:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 284 00000100:00100000:2.0:1787827241.324185:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488784896:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.324192:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->159743] (req 0->159743) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.324199:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->159743] (req 0->159743) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.324206:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.324214:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.324221:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.324226:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000070dc281b/0xebdcd48a4298ed79 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa1:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc8be expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.324247:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890a19400) 00000100:00100000:2.0:1787827241.324287:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488784896:12345-192.168.203.140@tcp:105: Request processed in 100us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.324304:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 284 00000100:00100000:2.0:1787827241.324996:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476632320:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.333050:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476633216:192.168.203.140@tcp:101:cp.0 00000080:00200000:2.0:1787827241.333973:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x42:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.333980:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.333985:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.333987:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2fe30 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.333991:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.333993:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2fe30 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.334002:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x42:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.334077:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-pfl.sh:[0x240001b71:0x42:0x0], ppos: 0, count: 103076 00000080:00200000:2.0:1787827241.334084:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-pfl.sh: write ppos: 0, bytes: 103076 00000080:00200000:2.0:1787827241.334089:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-pfl.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.334094:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@000000009df376cd[0x110, 1, [0x240001b71:0x42:0x0] hash] 00000020:00200000:2.0:1787827241.334096:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 103076) 0 0 00000080:00200000:2.0:1787827241.334098:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x42:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.334106:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 25] 00020000:00200000:2.0:1787827241.334113:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 103076) 103076, 103076 00020000:00200000:2.0:1787827241.334117:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.334144:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 25] 00000008:00010000:2.0:1787827241.334156:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.334160:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c430, osc lock ffff9a7884062888, flags 0x0 00010000:00010000:2.0:1787827241.334168:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/162 (0 106495) 00010000:00010000:2.0:1787827241.334189:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.334197:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->106495] (req 0->106495) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.334204:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->106495] (req 0->106495) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.334214:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476633344:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.334246:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.335091:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785024 00000100:00100000:1.0:1787827241.335097:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785024 00000100:00100000:1.0:1787827241.335140:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 285 00000100:00100000:1.0:1787827241.335149:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838ca940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488785024:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.335160:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003664abf3/0xebdcd48a4298ed95 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc912 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.335206:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.335211:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838ca940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488785024:12345-192.168.203.140@tcp:104: Request processed in 64us (166us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.335219:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 285 00000100:00100000:2.0:1787827241.335880:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476633344:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.335897:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->106495] (req 0->106495) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.335909:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->106495] (req 0->106495) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.335917:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->106495] (req 0->106495) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.336264:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785152 00000100:00100000:2.0:1787827241.336268:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785152 00000100:00100000:2.0:1787827241.336297:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 286 00000100:00100000:2.0:1787827241.336305:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488785152:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.336315:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->106495] (req 0->106495) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.336333:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->106495] (req 0->106495) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.336340:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->106495) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.336347:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->106495) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.336353:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->106495) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.336360:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a7c1e06f/0xebdcd48a4298ed9c lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->106495) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc919 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.336379:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890907c00) 00000100:00100000:2.0:1787827241.336416:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488785152:12345-192.168.203.140@tcp:105: Request processed in 112us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.336424:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 286 00000100:00100000:1.0:1787827241.337088:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476633472:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.345263:0:48371:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787827241.345278:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785280 00000100:00100000:1.0:1787827241.345284:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785280 00000100:00100000:1.0:1787827241.345322:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 287 00000100:00100000:1.0:1787827241.345331:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a7885af3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488785280:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.345341:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298edb8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc96d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.345385:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.345390:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a7885af3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488785280:12345-192.168.203.140@tcp:104: Request processed in 62us (202us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.345397:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 287 00010000:00010000:2.0:1787827241.345416:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827241.345424:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298edb8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc96d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.345435:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a18000 already unused, calling callback (ffffffffc125b410) 00010000:00010000:2.0:1787827241.345442:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298edb8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dc96d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.345460:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x43:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.345470:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x43:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827241.345516:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298edb8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc96d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.345527:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.345550:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298edb8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc96d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.345561:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298edb8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dc96d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.345587:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.345590:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.345593:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.346056:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667c3c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476634496:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.346067:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->258047] (req 0->258047) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346075:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->258047] (req 0->258047) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346082:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->258047] (req 0->258047) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.346321:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785408 00000100:00100000:2.0:1787827241.346327:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785408 00000100:00100000:2.0:1787827241.346363:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 288 00000100:00100000:2.0:1787827241.346371:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75570:x1874671488785408:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.346378:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->258047] (req 0->258047) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346385:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->258047] (req 0->258047) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346392:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346399:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346405:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346410:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000006d897692/0xebdcd48a4298edbf lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc974 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.346426:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890a19000) 00000100:00100000:2.0:1787827241.346455:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75570:x1874671488785408:12345-192.168.203.140@tcp:105: Request processed in 83us (166us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.346468:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 288 00000100:00100000:2.0:1787827241.347032:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476634624:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.354723:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476635520:192.168.203.140@tcp:101:cp.0 00000080:00200000:2.0:1787827241.355586:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x44:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.355593:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.355597:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.355600:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2fab0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.355603:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.355606:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2fab0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.355617:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x44:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.355666:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-scrub.sh:[0x240001b71:0x44:0x0], ppos: 0, count: 42517 00000080:00200000:2.0:1787827241.355673:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-scrub.sh: write ppos: 0, bytes: 42517 00000080:00200000:2.0:1787827241.355678:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-scrub.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.355683:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000e7034f4c[0x110, 1, [0x240001b71:0x44:0x0] hash] 00000020:00200000:2.0:1787827241.355685:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 42517) 0 0 00000080:00200000:2.0:1787827241.355687:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x44:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.355694:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 10] 00020000:00200000:2.0:1787827241.355702:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 42517) 42517, 42517 00020000:00200000:2.0:1787827241.355705:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.355715:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 10] 00000008:00010000:2.0:1787827241.355723:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.355725:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c130, osc lock ffff9a7884062c30, flags 0x0 00010000:00010000:2.0:1787827241.355731:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/163 (0 45055) 00010000:00010000:2.0:1787827241.355747:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.355753:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->45055] (req 0->45055) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.355763:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->45055] (req 0->45055) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.355773:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476635648:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.355803:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.356530:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785536 00000100:00100000:1.0:1787827241.356537:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785536 00000100:00100000:1.0:1787827241.356575:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 289 00000100:00100000:1.0:1787827241.356583:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838ca580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488785536:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.356595:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000005cfaa580/0xebdcd48a4298eddb lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dc9c8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.356641:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.356647:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838ca580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488785536:12345-192.168.203.140@tcp:104: Request processed in 66us (169us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.356656:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 289 00000100:00100000:2.0:1787827241.357733:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785664 00000100:00100000:2.0:1787827241.357739:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785664 00000100:00100000:2.0:1787827241.357765:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 290 00000100:00100000:2.0:1787827241.357771:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:78039:x1874671488785664:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.357778:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->45055] (req 0->45055) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc9cf expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.357786:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->45055] (req 0->45055) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dc9cf expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.357793:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->45055) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dc9cf expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.357799:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->45055) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc9cf expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.357805:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->45055) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dc9cf expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.357810:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000089b82b59/0xebdcd48a4298ede2 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->45055) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dc9cf expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.357826:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890907000) 00000100:00100000:2.0:1787827241.357889:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:78039:x1874671488785664:12345-192.168.203.140@tcp:105: Request processed in 115us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.357904:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 290 00000100:00100000:1.0:1787827241.358426:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476635776:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787827241.366198:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785792 00000100:00100000:1.0:1787827241.366204:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785792 00000100:00100000:1.0:1787827241.366240:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 291 00000100:00100000:1.0:1787827241.366247:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488785792:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.366256:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003664abf3/0xebdcd48a4298edfe lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dca23 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.366297:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.366303:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488785792:12345-192.168.203.140@tcp:104: Request processed in 57us (146us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.366309:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 291 00010000:00010000:2.0:1787827241.366335:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827241.366344:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003664abf3/0xebdcd48a4298edfe lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dca23 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.366356:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a18400 already unused, calling callback (ffffffffc125b410) 00010000:00010000:2.0:1787827241.366362:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003664abf3/0xebdcd48a4298edfe lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dca23 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.366378:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x45:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.366386:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x45:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827241.366414:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003664abf3/0xebdcd48a4298edfe lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dca23 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.366420:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.366442:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003664abf3/0xebdcd48a4298edfe lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dca23 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.366452:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003664abf3/0xebdcd48a4298edfe lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dca23 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.366474:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.366476:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.366478:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.366896:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667fc00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476636800:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.366909:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->438271] (req 0->438271) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.366918:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->438271] (req 0->438271) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.366925:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->438271] (req 0->438271) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.367248:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488785920 00000100:00100000:2.0:1787827241.367253:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488785920 00000100:00100000:2.0:1787827241.367284:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 292 00000100:00100000:2.0:1787827241.367292:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a667e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488785920:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.367302:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->438271] (req 0->438271) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.367313:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->438271] (req 0->438271) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.367323:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.367337:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.367343:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.367349:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.367368:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7884b12200) 00000100:00100000:2.0:1787827241.367401:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a667e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488785920:12345-192.168.203.140@tcp:105: Request processed in 110us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.367410:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 292 00000080:00200000:0.0:1787827241.367463:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) sanity-sec.sh: write [0, 434581) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.367475:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367516:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367525:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001a2451ab[3 0000000007b63142 2 1 00000000746d8de4 0 0 0] 00000080:00200000:0.0:1787827241.367528:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000031e68480 17ffffc0001001 3:0 ffff9a7884db91d0 0 lru 00000080:00200000:0.0:1787827241.367534:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000038b9dcd6 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367536:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001a2451ab 00000080:00200000:0.0:1787827241.367537:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.367540:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367557:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367560:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000091ab0479[3 0000000007b63142 2 1 00000000746d8de4 0 0 1] 00000080:00200000:0.0:1787827241.367563:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000dee99260 17ffffc0001001 3:0 ffff9a7884db9740 1 lru 00000080:00200000:0.0:1787827241.367567:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000447930c0 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367568:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000091ab0479 00000080:00200000:0.0:1787827241.367569:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:0.0:1787827241.367570:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367581:0:48385:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367584:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000039c869c2[3 0000000007b63142 2 1 00000000746d8de4 0 0 2] 00000080:00200000:0.0:1787827241.367585:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003a087da8 17ffffc0001001 3:0 ffff9a7884db9e80 2 lru 00000080:00200000:0.0:1787827241.367589:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000071bf197a 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367591:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000039c869c2 00000080:00200000:0.0:1787827241.367592:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:0.0:1787827241.367593:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367602:0:48385:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367605:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d82136ef[3 0000000007b63142 2 1 00000000746d8de4 0 0 3] 00000080:00200000:0.0:1787827241.367607:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ed8590c3 17ffffc0001001 3:0 ffff9a7884db9488 3 lru 00000080:00200000:0.0:1787827241.367610:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000dd56be6 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367612:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d82136ef 00000080:00200000:0.0:1787827241.367613:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:0.0:1787827241.367614:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367626:0:48385:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367629:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007a834270[3 0000000007b63142 2 1 00000000746d8de4 0 0 4] 00000080:00200000:0.0:1787827241.367630:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d6bae4b7 17ffffc0001021 2:0 ffff9a7884db9bc8 4 lru 00000080:00200000:0.0:1787827241.367635:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000df7e39fa 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367636:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007a834270 00000080:00200000:0.0:1787827241.367637:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:0.0:1787827241.367638:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367647:0:48385:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367650:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000038a01d76[3 0000000007b63142 2 1 00000000746d8de4 0 0 5] 00000080:00200000:0.0:1787827241.367651:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000a0799de 17ffffc0001001 3:0 ffff9a7884db9570 5 lru 00000080:00200000:0.0:1787827241.367655:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000050aa8fac 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367657:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000038a01d76 00000080:00200000:0.0:1787827241.367658:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:0.0:1787827241.367659:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367667:0:48385:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367669:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e4ce22dc[3 0000000007b63142 2 1 00000000746d8de4 0 0 6] 00000080:00200000:0.0:1787827241.367671:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000031756b21 17ffffc0001001 3:0 ffff9a7884db9828 6 lru 00000080:00200000:0.0:1787827241.367674:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ce30a16e 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367676:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e4ce22dc 00000080:00200000:0.0:1787827241.367677:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:0.0:1787827241.367678:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367686:0:48385:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367689:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000090e2ff2c[3 0000000007b63142 2 1 00000000746d8de4 0 0 7] 00000080:00200000:0.0:1787827241.367690:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004b402a67 17ffffc0001001 3:0 ffff9a7884db9000 7 lru 00000080:00200000:0.0:1787827241.367694:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009c845ca9 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367695:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000090e2ff2c 00000080:00200000:0.0:1787827241.367696:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:0.0:1787827241.367697:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367705:0:48385:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367707:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dd4b4244[3 0000000007b63142 2 1 00000000746d8de4 0 0 8] 00000080:00200000:0.0:1787827241.367709:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b91b4e5c 17ffffc0001001 3:0 ffff9a7884db9cb0 8 lru 00000080:00200000:0.0:1787827241.367712:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000066ec747e 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367713:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dd4b4244 00000080:00200000:0.0:1787827241.367714:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:0.0:1787827241.367716:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367726:0:48385:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367728:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008329bf49[3 0000000007b63142 2 1 00000000746d8de4 0 0 9] 00000080:00200000:0.0:1787827241.367730:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000023a21aa0 17ffffc0001001 3:0 ffff9a7884db9ae0 9 lru 00000080:00200000:0.0:1787827241.367733:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000056a84afa 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367735:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008329bf49 00000080:00200000:0.0:1787827241.367736:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:0.0:1787827241.367737:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367745:0:48385:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367748:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000053447fec[3 0000000007b63142 2 1 00000000746d8de4 0 0 10] 00000080:00200000:0.0:1787827241.367750:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000039261118 17ffffc0001001 3:0 ffff9a7884db9658 10 lru 00000080:00200000:0.0:1787827241.367754:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000090d271fb 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367755:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000053447fec 00000080:00200000:0.0:1787827241.367756:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:0.0:1787827241.367757:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367765:0:48385:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367768:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000da6b4e1[3 0000000007b63142 2 1 00000000746d8de4 0 0 11] 00000080:00200000:0.0:1787827241.367770:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000960a4fae 17ffffc0001001 3:0 ffff9a7884db99f8 11 lru 00000080:00200000:0.0:1787827241.367773:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c5be39f1 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367775:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000da6b4e1 00000080:00200000:0.0:1787827241.367775:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:0.0:1787827241.367777:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367785:0:48385:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367788:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000950e0907[3 0000000007b63142 2 1 00000000746d8de4 0 0 12] 00000080:00200000:0.0:1787827241.367790:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c2f806f6 17ffffc0001001 3:0 ffff9a7884db9910 12 lru 00000080:00200000:0.0:1787827241.367793:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005c49ba8e 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367795:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000950e0907 00000080:00200000:0.0:1787827241.367796:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:0.0:1787827241.367797:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367814:0:48385:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367816:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a4e20585[3 0000000007b63142 2 1 00000000746d8de4 0 0 13] 00000080:00200000:0.0:1787827241.367818:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000d493d54 17ffffc0001001 3:0 ffff9a7888376658 13 lru 00000080:00200000:0.0:1787827241.367823:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f273b288 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367824:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a4e20585 00000080:00200000:0.0:1787827241.367825:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:0.0:1787827241.367826:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367869:0:48385:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367876:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000862345a[3 0000000007b63142 2 1 00000000746d8de4 0 0 14] 00000080:00200000:0.0:1787827241.367880:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000066fc77d9 17ffffc0001001 3:0 ffff9a78883769f8 14 lru 00000080:00200000:0.0:1787827241.367886:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000066bd9446 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367888:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000862345a 00000080:00200000:0.0:1787827241.367889:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:0.0:1787827241.367891:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367903:0:48385:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367907:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ae788a19[3 0000000007b63142 2 1 00000000746d8de4 0 0 15] 00000080:00200000:0.0:1787827241.367909:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000186dbac3 17ffffc0001001 3:0 ffff9a7888376910 15 lru 00000080:00200000:0.0:1787827241.367913:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000018fe0d1 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367916:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ae788a19 00000080:00200000:0.0:1787827241.367917:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:0.0:1787827241.367918:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367929:0:48385:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367933:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000025d69d3b[3 0000000007b63142 2 1 00000000746d8de4 0 0 16] 00000080:00200000:0.0:1787827241.367935:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a8985237 17ffffc0001001 3:0 ffff9a7888376d98 16 lru 00000080:00200000:0.0:1787827241.367939:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000060b92fcc 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367941:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000025d69d3b 00000080:00200000:0.0:1787827241.367942:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000080:00200000:0.0:1787827241.367944:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.367955:0:48385:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:0.0:1787827241.367968:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000edc04ff5[3 0000000007b63142 2 1 00000000746d8de4 0 0 17] 00000080:00200000:0.0:1787827241.367972:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001988e69c 17ffffc0001001 3:0 ffff9a78883763a0 17 lru 00000080:00200000:0.0:1787827241.367978:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000ae4d608 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.367981:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000edc04ff5 00000080:00200000:0.0:1787827241.367982:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:0.0:1787827241.367985:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000100:00100000:2.0:1787827241.367997:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476636928:192.168.203.140@tcp:103:ldlm_bl.0 00000080:00200000:0.0:1787827241.368001:0:48385:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368005:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a8ff7cba[3 0000000007b63142 2 1 00000000746d8de4 0 0 18] 00000080:00200000:0.0:1787827241.368008:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f21c75b9 17ffffc0001001 3:0 ffff9a78883760e8 18 lru 00000080:00200000:0.0:1787827241.368014:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000ca68ef7 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368017:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a8ff7cba 00000080:00200000:0.0:1787827241.368019:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:0.0:1787827241.368021:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368040:0:48385:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368044:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002b5271fd[3 0000000007b63142 2 1 00000000746d8de4 0 0 19] 00000080:00200000:0.0:1787827241.368046:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000650b3c44 17ffffc0001021 2:0 ffff9a78883762b8 19 lru 00000080:00200000:0.0:1787827241.368050:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000026cf1fe5 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368052:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002b5271fd 00000080:00200000:0.0:1787827241.368053:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:0.0:1787827241.368054:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368064:0:48385:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368066:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f1f6ecae[3 0000000007b63142 2 1 00000000746d8de4 0 0 20] 00000080:00200000:0.0:1787827241.368068:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002ad6a227 17ffffc0001001 3:0 ffff9a78883761d0 20 lru 00000080:00200000:0.0:1787827241.368072:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000020c706ad 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368074:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f1f6ecae 00000080:00200000:0.0:1787827241.368075:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:0.0:1787827241.368076:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368084:0:48385:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368087:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000015963fec[3 0000000007b63142 2 1 00000000746d8de4 0 0 21] 00000080:00200000:0.0:1787827241.368089:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007747dd7e 17ffffc0001001 3:0 ffff9a7888376740 21 lru 00000080:00200000:0.0:1787827241.368093:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008e29cb20 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368094:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000015963fec 00000080:00200000:0.0:1787827241.368095:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:0.0:1787827241.368096:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368105:0:48385:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368107:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cfda7360[3 0000000007b63142 2 1 00000000746d8de4 0 0 22] 00000080:00200000:0.0:1787827241.368109:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000104c5ac1 17ffffc0001001 3:0 ffff9a7888376e80 22 lru 00000080:00200000:0.0:1787827241.368113:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000c817d78 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368115:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cfda7360 00000080:00200000:0.0:1787827241.368116:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:0.0:1787827241.368117:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368127:0:48385:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368130:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001f22474b[3 0000000007b63142 2 1 00000000746d8de4 0 0 23] 00000080:00200000:0.0:1787827241.368132:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c16aa7df 17ffffc0001001 3:0 ffff9a7888376488 23 lru 00000080:00200000:0.0:1787827241.368135:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003651d9fd 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368137:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001f22474b 00000080:00200000:0.0:1787827241.368138:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:0.0:1787827241.368139:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368147:0:48385:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368150:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a5d1cb62[3 0000000007b63142 2 1 00000000746d8de4 0 0 24] 00000080:00200000:0.0:1787827241.368152:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000054b7e6a5 17ffffc0001001 3:0 ffff9a7888376bc8 24 lru 00000080:00200000:0.0:1787827241.368155:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000030e6e0e5 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368156:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a5d1cb62 00000080:00200000:0.0:1787827241.368157:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:0.0:1787827241.368158:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368166:0:48385:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368169:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bdbdc2cc[3 0000000007b63142 2 1 00000000746d8de4 0 0 25] 00000080:00200000:0.0:1787827241.368170:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000069d0d56f 17ffffc0001001 3:0 ffff9a7888376570 25 lru 00000080:00200000:0.0:1787827241.368174:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000099d98074 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368175:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bdbdc2cc 00000080:00200000:0.0:1787827241.368176:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:0.0:1787827241.368177:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368184:0:48385:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368187:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002e323478[3 0000000007b63142 2 1 00000000746d8de4 0 0 26] 00000080:00200000:0.0:1787827241.368188:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002d0dbe4d 17ffffc0001001 3:0 ffff9a7888376828 26 lru 00000080:00200000:0.0:1787827241.368192:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b4d35e22 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368194:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002e323478 00000080:00200000:0.0:1787827241.368195:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:0.0:1787827241.368196:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368204:0:48385:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368207:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000014f7f826[3 0000000007b63142 2 1 00000000746d8de4 0 0 27] 00000080:00200000:0.0:1787827241.368209:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000459232b7 17ffffc0001001 3:0 ffff9a7888376000 27 lru 00000080:00200000:0.0:1787827241.368212:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c45ecf89 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368214:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000014f7f826 00000080:00200000:0.0:1787827241.368215:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:0.0:1787827241.368216:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368224:0:48385:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368227:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003c731029[3 0000000007b63142 2 1 00000000746d8de4 0 0 28] 00000080:00200000:0.0:1787827241.368228:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000e1b83d6 17ffffc0001001 3:0 ffff9a7888376cb0 28 lru 00000080:00200000:0.0:1787827241.368232:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000022177907 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368233:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003c731029 00000080:00200000:0.0:1787827241.368234:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:0.0:1787827241.368235:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368243:0:48385:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368245:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e42e55c3[3 0000000007b63142 2 1 00000000746d8de4 0 0 29] 00000080:00200000:0.0:1787827241.368247:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003861b128 17ffffc0001001 3:0 ffff9a7888376ae0 29 lru 00000080:00200000:0.0:1787827241.368251:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b46c05a4 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368252:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e42e55c3 00000080:00200000:0.0:1787827241.368253:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:0.0:1787827241.368254:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368269:0:48385:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368272:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004d3686a1[3 0000000007b63142 2 1 00000000746d8de4 0 0 30] 00000080:00200000:0.0:1787827241.368274:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cf69cd5c 17ffffc0001001 3:0 ffff9a7890a94740 30 lru 00000080:00200000:0.0:1787827241.368277:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d2d93056 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368278:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004d3686a1 00000080:00200000:0.0:1787827241.368279:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:0.0:1787827241.368280:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368289:0:48385:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368291:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000778e68fb[3 0000000007b63142 2 1 00000000746d8de4 0 0 31] 00000080:00200000:0.0:1787827241.368293:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000384be7e4 17ffffc0001001 3:0 ffff9a7890a94e80 31 lru 00000080:00200000:0.0:1787827241.368297:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e049c115 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368299:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000778e68fb 00000080:00200000:0.0:1787827241.368300:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:0.0:1787827241.368301:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368309:0:48385:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368312:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006ae03ae0[3 0000000007b63142 2 1 00000000746d8de4 0 0 32] 00000080:00200000:0.0:1787827241.368313:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008a3c4ed8 17ffffc0001001 3:0 ffff9a7890a94488 32 lru 00000080:00200000:0.0:1787827241.368317:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001a71f4c1 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368319:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006ae03ae0 00000080:00200000:0.0:1787827241.368320:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:0.0:1787827241.368321:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368329:0:48385:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368332:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ca77a022[3 0000000007b63142 2 1 00000000746d8de4 0 0 33] 00000080:00200000:0.0:1787827241.368333:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a42d8c14 17ffffc0001001 3:0 ffff9a7890a94bc8 33 lru 00000080:00200000:0.0:1787827241.368337:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005a0d96a2 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368338:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ca77a022 00000080:00200000:0.0:1787827241.368339:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:0.0:1787827241.368340:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368350:0:48385:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368352:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000093cfb8e[3 0000000007b63142 2 1 00000000746d8de4 0 0 34] 00000080:00200000:0.0:1787827241.368354:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005d921438 17ffffc0001021 2:0 ffff9a7890a94570 34 lru 00000080:00200000:0.0:1787827241.368358:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fd5ff1c9 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368359:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000093cfb8e 00000080:00200000:0.0:1787827241.368359:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:0.0:1787827241.368360:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368368:0:48385:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368371:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009dc8dd1f[3 0000000007b63142 2 1 00000000746d8de4 0 0 35] 00000080:00200000:0.0:1787827241.368372:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000025e4b367 17ffffc0001001 3:0 ffff9a7890a94828 35 lru 00000080:00200000:0.0:1787827241.368376:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000008386ee5 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368378:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009dc8dd1f 00000080:00200000:0.0:1787827241.368379:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:0.0:1787827241.368380:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368388:0:48385:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368391:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002d435a30[3 0000000007b63142 2 1 00000000746d8de4 0 0 36] 00000080:00200000:0.0:1787827241.368393:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000068a8378d 17ffffc0001001 3:0 ffff9a7890a94000 36 lru 00000080:00200000:0.0:1787827241.368396:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000066c9d54c 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368398:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002d435a30 00000080:00200000:0.0:1787827241.368399:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:0.0:1787827241.368400:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368410:0:48385:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368412:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003488b2ef[3 0000000007b63142 2 1 00000000746d8de4 0 0 37] 00000080:00200000:0.0:1787827241.368414:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d589746a 17ffffc0001001 3:0 ffff9a7890a94cb0 37 lru 00000080:00200000:0.0:1787827241.368417:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009ab94c37 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368418:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003488b2ef 00000080:00200000:0.0:1787827241.368419:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:0.0:1787827241.368420:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368429:0:48385:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368432:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f96e3915[3 0000000007b63142 2 1 00000000746d8de4 0 0 38] 00000080:00200000:0.0:1787827241.368434:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000edaad336 17ffffc0001001 3:0 ffff9a7890a94ae0 38 lru 00000080:00200000:0.0:1787827241.368437:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000035c9a027 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368439:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f96e3915 00000080:00200000:0.0:1787827241.368440:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:0.0:1787827241.368441:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368448:0:48385:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368451:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d9eb90eb[3 0000000007b63142 2 1 00000000746d8de4 0 0 39] 00000080:00200000:0.0:1787827241.368452:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005e555c3d 17ffffc0001001 3:0 ffff9a7890a94658 39 lru 00000080:00200000:0.0:1787827241.368456:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000295f57b1 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368457:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d9eb90eb 00000080:00200000:0.0:1787827241.368457:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:0.0:1787827241.368458:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368466:0:48385:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368469:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000164c8d9d[3 0000000007b63142 2 1 00000000746d8de4 0 0 40] 00000080:00200000:0.0:1787827241.368471:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bf6222f0 17ffffc0001001 3:0 ffff9a7890a949f8 40 lru 00000080:00200000:0.0:1787827241.368474:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000000ddcf0b 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368475:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000164c8d9d 00000080:00200000:0.0:1787827241.368476:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:0.0:1787827241.368477:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368485:0:48385:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368487:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e3d2474c[3 0000000007b63142 2 1 00000000746d8de4 0 0 41] 00000080:00200000:0.0:1787827241.368489:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000097352719 17ffffc0001001 3:0 ffff9a7890a94910 41 lru 00000080:00200000:0.0:1787827241.368493:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f7fad52b 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368494:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e3d2474c 00000080:00200000:0.0:1787827241.368495:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:0.0:1787827241.368496:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368518:0:48385:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368522:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002bcf4239[3 0000000007b63142 2 1 00000000746d8de4 0 0 42] 00000080:00200000:0.0:1787827241.368524:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d63bd1c7 17ffffc0001001 3:0 ffff9a7890a94d98 42 lru 00000080:00200000:0.0:1787827241.368527:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f150c9a6 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368529:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002bcf4239 00000080:00200000:0.0:1787827241.368530:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:0.0:1787827241.368531:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368542:0:48385:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368545:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004c5f8daf[3 0000000007b63142 2 1 00000000746d8de4 0 0 43] 00000080:00200000:0.0:1787827241.368547:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004567b0c7 17ffffc0001001 3:0 ffff9a7890a943a0 43 lru 00000080:00200000:0.0:1787827241.368551:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007048f7db 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368552:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004c5f8daf 00000080:00200000:0.0:1787827241.368553:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:0.0:1787827241.368554:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368564:0:48385:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368566:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a751691e[3 0000000007b63142 2 1 00000000746d8de4 0 0 44] 00000080:00200000:0.0:1787827241.368568:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000076b34628 17ffffc0001001 3:0 ffff9a7890a940e8 44 lru 00000080:00200000:0.0:1787827241.368572:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b9b7b796 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368573:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a751691e 00000080:00200000:0.0:1787827241.368574:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:0.0:1787827241.368575:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368584:0:48385:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368586:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ad74efea[3 0000000007b63142 2 1 00000000746d8de4 0 0 45] 00000080:00200000:0.0:1787827241.368588:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fc8d9e8c 17ffffc0001001 3:0 ffff9a7890a942b8 45 lru 00000080:00200000:0.0:1787827241.368592:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f674dd27 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368594:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ad74efea 00000080:00200000:0.0:1787827241.368594:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:0.0:1787827241.368595:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368603:0:48385:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368605:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009930021a[3 0000000007b63142 2 1 00000000746d8de4 0 0 46] 00000080:00200000:0.0:1787827241.368607:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003460d620 17ffffc0001001 3:0 ffff9a7890a941d0 46 lru 00000080:00200000:0.0:1787827241.368610:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005b54f558 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368612:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009930021a 00000080:00200000:0.0:1787827241.368612:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:0.0:1787827241.368613:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368628:0:48385:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368631:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000002eec72d[3 0000000007b63142 2 1 00000000746d8de4 0 0 47] 00000080:00200000:0.0:1787827241.368632:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001a7549bd 17ffffc0001001 3:0 ffff9a788904ad98 47 lru 00000080:00200000:0.0:1787827241.368636:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000030b85632 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368637:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000002eec72d 00000080:00200000:0.0:1787827241.368638:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:0.0:1787827241.368639:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368647:0:48385:0:(rw26.c:875:ll_write_end()) pos 196608, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368650:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001819f094[3 0000000007b63142 2 1 00000000746d8de4 0 0 48] 00000080:00200000:0.0:1787827241.368651:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000065fd6acc 17ffffc0001001 3:0 ffff9a788904a3a0 48 lru 00000080:00200000:0.0:1787827241.368655:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ad319398 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368656:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001819f094 00000080:00200000:0.0:1787827241.368657:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:0.0:1787827241.368658:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 49 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368670:0:48385:0:(rw26.c:875:ll_write_end()) pos 200704, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368673:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a5e49b3b[3 0000000007b63142 2 1 00000000746d8de4 0 0 49] 00000080:00200000:0.0:1787827241.368675:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000da394246 17ffffc0001021 2:0 ffff9a788904a0e8 49 lru 00000080:00200000:0.0:1787827241.368679:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a6d8e939 49: 1< 0 - - > 2< 200704 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368681:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a5e49b3b 00000080:00200000:0.0:1787827241.368682:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 50. 00000080:00200000:0.0:1787827241.368683:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 50 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368691:0:48385:0:(rw26.c:875:ll_write_end()) pos 204800, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368694:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cb3a98fc[3 0000000007b63142 2 1 00000000746d8de4 0 0 50] 00000080:00200000:0.0:1787827241.368695:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000969f90e6 17ffffc0001001 3:0 ffff9a788904a2b8 50 lru 00000080:00200000:0.0:1787827241.368699:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000363cf888 50: 1< 0 - - > 2< 204800 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368701:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cb3a98fc 00000080:00200000:0.0:1787827241.368702:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 51. 00000080:00200000:0.0:1787827241.368703:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 51 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368712:0:48385:0:(rw26.c:875:ll_write_end()) pos 208896, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368715:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000086a0e65a[3 0000000007b63142 2 1 00000000746d8de4 0 0 51] 00000080:00200000:0.0:1787827241.368716:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002ea8496f 17ffffc0001001 3:0 ffff9a788904a1d0 51 lru 00000080:00200000:0.0:1787827241.368720:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000324fcdb2 51: 1< 0 - - > 2< 208896 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368722:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000086a0e65a 00000080:00200000:0.0:1787827241.368723:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 52. 00000080:00200000:0.0:1787827241.368724:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 52 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368732:0:48385:0:(rw26.c:875:ll_write_end()) pos 212992, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368735:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009caca2f8[3 0000000007b63142 2 1 00000000746d8de4 0 0 52] 00000080:00200000:0.0:1787827241.368737:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009705d9a9 17ffffc0001001 3:0 ffff9a788904a740 52 lru 00000080:00200000:0.0:1787827241.368740:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009d8246db 52: 1< 0 - - > 2< 212992 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368742:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009caca2f8 00000080:00200000:0.0:1787827241.368743:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 53. 00000080:00200000:0.0:1787827241.368744:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 53 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368752:0:48385:0:(rw26.c:875:ll_write_end()) pos 217088, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368755:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c36a2ba6[3 0000000007b63142 2 1 00000000746d8de4 0 0 53] 00000080:00200000:0.0:1787827241.368756:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002888d612 17ffffc0001001 3:0 ffff9a788904ae80 53 lru 00000080:00200000:0.0:1787827241.368760:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000164e9b65 53: 1< 0 - - > 2< 217088 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368762:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c36a2ba6 00000080:00200000:0.0:1787827241.368763:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 54. 00000080:00200000:0.0:1787827241.368764:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 54 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368772:0:48385:0:(rw26.c:875:ll_write_end()) pos 221184, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368775:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003c62670c[3 0000000007b63142 2 1 00000000746d8de4 0 0 54] 00000080:00200000:0.0:1787827241.368777:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000354965ba 17ffffc0001001 3:0 ffff9a788904a488 54 lru 00000080:00200000:0.0:1787827241.368780:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004f4f4ca4 54: 1< 0 - - > 2< 221184 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368781:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003c62670c 00000080:00200000:0.0:1787827241.368782:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 55. 00000080:00200000:0.0:1787827241.368783:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 55 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368791:0:48385:0:(rw26.c:875:ll_write_end()) pos 225280, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368793:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b9d19547[3 0000000007b63142 2 1 00000000746d8de4 0 0 55] 00000080:00200000:0.0:1787827241.368795:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b5d87787 17ffffc0001001 3:0 ffff9a788904abc8 55 lru 00000080:00200000:0.0:1787827241.368798:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000080c04c4a 55: 1< 0 - - > 2< 225280 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368799:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b9d19547 00000080:00200000:0.0:1787827241.368800:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 56. 00000080:00200000:0.0:1787827241.368801:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 56 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368810:0:48385:0:(rw26.c:875:ll_write_end()) pos 229376, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368812:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001b85c5e8[3 0000000007b63142 2 1 00000000746d8de4 0 0 56] 00000080:00200000:0.0:1787827241.368814:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004566c298 17ffffc0001001 3:0 ffff9a788904a570 56 lru 00000080:00200000:0.0:1787827241.368818:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ab08ff4d 56: 1< 0 - - > 2< 229376 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368819:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001b85c5e8 00000080:00200000:0.0:1787827241.368820:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 57. 00000080:00200000:0.0:1787827241.368821:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 57 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368847:0:48385:0:(rw26.c:875:ll_write_end()) pos 233472, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368853:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000068624ba8[3 0000000007b63142 2 1 00000000746d8de4 0 0 57] 00000080:00200000:0.0:1787827241.368856:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000f51035b 17ffffc0001001 3:0 ffff9a788904a828 57 lru 00000080:00200000:0.0:1787827241.368863:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000343b9e6c 57: 1< 0 - - > 2< 233472 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368866:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000068624ba8 00000080:00200000:0.0:1787827241.368868:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 58. 00000080:00200000:0.0:1787827241.368870:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 58 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368889:0:48385:0:(rw26.c:875:ll_write_end()) pos 237568, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368894:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000056b38c39[3 0000000007b63142 2 1 00000000746d8de4 0 0 58] 00000080:00200000:0.0:1787827241.368897:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ed6c6856 17ffffc0001001 3:0 ffff9a788904a000 58 lru 00000080:00200000:0.0:1787827241.368901:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000031165b5c 58: 1< 0 - - > 2< 237568 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368903:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000056b38c39 00000080:00200000:0.0:1787827241.368904:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 59. 00000080:00200000:0.0:1787827241.368906:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 59 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368917:0:48385:0:(rw26.c:875:ll_write_end()) pos 241664, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368919:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000032cb482b[3 0000000007b63142 2 1 00000000746d8de4 0 0 59] 00000080:00200000:0.0:1787827241.368921:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000392b0da4 17ffffc0001001 3:0 ffff9a788904acb0 59 lru 00000080:00200000:0.0:1787827241.368925:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006339828d 59: 1< 0 - - > 2< 241664 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368927:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000032cb482b 00000080:00200000:0.0:1787827241.368928:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 60. 00000080:00200000:0.0:1787827241.368929:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 60 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368937:0:48385:0:(rw26.c:875:ll_write_end()) pos 245760, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368940:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000025731b2d[3 0000000007b63142 2 1 00000000746d8de4 0 0 60] 00000080:00200000:0.0:1787827241.368942:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000011e71d6d 17ffffc0001001 3:0 ffff9a788904aae0 60 lru 00000080:00200000:0.0:1787827241.368945:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b042cd4c 60: 1< 0 - - > 2< 245760 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368947:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000025731b2d 00000080:00200000:0.0:1787827241.368948:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 61. 00000080:00200000:0.0:1787827241.368949:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 61 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368957:0:48385:0:(rw26.c:875:ll_write_end()) pos 249856, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368959:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001c0a709c[3 0000000007b63142 2 1 00000000746d8de4 0 0 61] 00000080:00200000:0.0:1787827241.368961:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a99720c9 17ffffc0001001 3:0 ffff9a788904a658 61 lru 00000080:00200000:0.0:1787827241.368965:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000086af5fe3 61: 1< 0 - - > 2< 249856 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368966:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001c0a709c 00000080:00200000:0.0:1787827241.368966:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 62. 00000080:00200000:0.0:1787827241.368967:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 62 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368975:0:48385:0:(rw26.c:875:ll_write_end()) pos 253952, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368978:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a207b724[3 0000000007b63142 2 1 00000000746d8de4 0 0 62] 00000080:00200000:0.0:1787827241.368980:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ec817cf3 17ffffc0001001 3:0 ffff9a788904a9f8 62 lru 00000080:00200000:0.0:1787827241.368983:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002bd6a7d5 62: 1< 0 - - > 2< 253952 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.368984:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a207b724 00000080:00200000:0.0:1787827241.368985:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 63. 00000080:00200000:0.0:1787827241.368986:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 63 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.368994:0:48385:0:(rw26.c:875:ll_write_end()) pos 258048, len 4096, copied 4096 00000080:00200000:0.0:1787827241.368997:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e1974b0b[3 0000000007b63142 2 1 00000000746d8de4 0 0 63] 00000080:00200000:0.0:1787827241.368999:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000077573f19 17ffffc0001001 3:0 ffff9a788904a910 63 lru 00000080:00200000:0.0:1787827241.369002:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003292d431 63: 1< 0 - - > 2< 258048 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369004:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e1974b0b 00000080:00200000:0.0:1787827241.369004:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 64. 00000080:00200000:0.0:1787827241.369005:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 64 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369026:0:48385:0:(rw26.c:875:ll_write_end()) pos 262144, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369029:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cbfa3f75[3 0000000007b63142 2 1 00000000746d8de4 0 0 64] 00000080:00200000:0.0:1787827241.369031:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ca46991b 17ffffc0001021 2:0 ffff9a7885aa8828 64 lru 00000080:00200000:0.0:1787827241.369035:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e6c2f434 64: 1< 0 - - > 2< 262144 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369036:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cbfa3f75 00000080:00200000:0.0:1787827241.369036:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 65. 00000080:00200000:0.0:1787827241.369037:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 65 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369047:0:48385:0:(rw26.c:875:ll_write_end()) pos 266240, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369050:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000029141091[3 0000000007b63142 2 1 00000000746d8de4 0 0 65] 00000080:00200000:0.0:1787827241.369052:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004f3873f5 17ffffc0001001 3:0 ffff9a7885aa8000 65 lru 00000080:00200000:0.0:1787827241.369055:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001c849c71 65: 1< 0 - - > 2< 266240 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369057:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000029141091 00000080:00200000:0.0:1787827241.369058:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 66. 00000080:00200000:0.0:1787827241.369059:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 66 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369067:0:48385:0:(rw26.c:875:ll_write_end()) pos 270336, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369070:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003bd89491[3 0000000007b63142 2 1 00000000746d8de4 0 0 66] 00000080:00200000:0.0:1787827241.369072:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009246a548 17ffffc0001001 3:0 ffff9a7885aa8cb0 66 lru 00000080:00200000:0.0:1787827241.369076:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000088cfaba1 66: 1< 0 - - > 2< 270336 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369077:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003bd89491 00000080:00200000:0.0:1787827241.369078:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 67. 00000080:00200000:0.0:1787827241.369079:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 67 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369087:0:48385:0:(rw26.c:875:ll_write_end()) pos 274432, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369090:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000006a032fb[3 0000000007b63142 2 1 00000000746d8de4 0 0 67] 00000080:00200000:0.0:1787827241.369092:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002b145309 17ffffc0001001 3:0 ffff9a7885aa8ae0 67 lru 00000080:00200000:0.0:1787827241.369096:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f48db6ea 67: 1< 0 - - > 2< 274432 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369097:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000006a032fb 00000080:00200000:0.0:1787827241.369098:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 68. 00000080:00200000:0.0:1787827241.369099:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 68 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369107:0:48385:0:(rw26.c:875:ll_write_end()) pos 278528, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369110:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000824da6d4[3 0000000007b63142 2 1 00000000746d8de4 0 0 68] 00000080:00200000:0.0:1787827241.369111:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004ee49d28 17ffffc0001001 3:0 ffff9a7885aa8658 68 lru 00000080:00200000:0.0:1787827241.369115:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d6f1e2f1 68: 1< 0 - - > 2< 278528 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369116:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000824da6d4 00000080:00200000:0.0:1787827241.369117:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 69. 00000080:00200000:0.0:1787827241.369118:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 69 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369127:0:48385:0:(rw26.c:875:ll_write_end()) pos 282624, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369130:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fe21ede0[3 0000000007b63142 2 1 00000000746d8de4 0 0 69] 00000080:00200000:0.0:1787827241.369132:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007f8044d3 17ffffc0001001 3:0 ffff9a7885aa89f8 69 lru 00000080:00200000:0.0:1787827241.369136:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000eb63f440 69: 1< 0 - - > 2< 282624 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369137:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fe21ede0 00000080:00200000:0.0:1787827241.369138:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 70. 00000080:00200000:0.0:1787827241.369139:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 70 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369147:0:48385:0:(rw26.c:875:ll_write_end()) pos 286720, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369149:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a5a60336[3 0000000007b63142 2 1 00000000746d8de4 0 0 70] 00000080:00200000:0.0:1787827241.369151:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ff216515 17ffffc0001001 3:0 ffff9a7885aa8910 70 lru 00000080:00200000:0.0:1787827241.369155:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000220f9bad 70: 1< 0 - - > 2< 286720 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369156:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a5a60336 00000080:00200000:0.0:1787827241.369156:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 71. 00000080:00200000:0.0:1787827241.369157:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 71 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369165:0:48385:0:(rw26.c:875:ll_write_end()) pos 290816, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369168:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000045c075e6[3 0000000007b63142 2 1 00000000746d8de4 0 0 71] 00000080:00200000:0.0:1787827241.369169:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a7fc9c48 17ffffc0001001 3:0 ffff9a7885aa8d98 71 lru 00000080:00200000:0.0:1787827241.369173:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005bb05b11 71: 1< 0 - - > 2< 290816 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369174:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000045c075e6 00000080:00200000:0.0:1787827241.369175:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 72. 00000080:00200000:0.0:1787827241.369176:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 72 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369185:0:48385:0:(rw26.c:875:ll_write_end()) pos 294912, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369187:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e9720c26[3 0000000007b63142 2 1 00000000746d8de4 0 0 72] 00000080:00200000:0.0:1787827241.369189:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001149f57f 17ffffc0001001 3:0 ffff9a7885aa83a0 72 lru 00000080:00200000:0.0:1787827241.369193:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000016afaf7e 72: 1< 0 - - > 2< 294912 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369194:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e9720c26 00000080:00200000:0.0:1787827241.369195:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 73. 00000080:00200000:0.0:1787827241.369196:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 73 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369205:0:48385:0:(rw26.c:875:ll_write_end()) pos 299008, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369208:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000aabb7a00[3 0000000007b63142 2 1 00000000746d8de4 0 0 73] 00000080:00200000:0.0:1787827241.369210:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001ff8c86f 17ffffc0001001 3:0 ffff9a7885aa80e8 73 lru 00000080:00200000:0.0:1787827241.369214:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000eeac97a4 73: 1< 0 - - > 2< 299008 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369215:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000aabb7a00 00000080:00200000:0.0:1787827241.369216:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 74. 00000080:00200000:0.0:1787827241.369217:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 74 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369225:0:48385:0:(rw26.c:875:ll_write_end()) pos 303104, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369227:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b1c92edd[3 0000000007b63142 2 1 00000000746d8de4 0 0 74] 00000080:00200000:0.0:1787827241.369229:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008dbed6e2 17ffffc0001001 3:0 ffff9a7885aa82b8 74 lru 00000080:00200000:0.0:1787827241.369233:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000096efe935 74: 1< 0 - - > 2< 303104 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369234:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b1c92edd 00000080:00200000:0.0:1787827241.369235:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 75. 00000080:00200000:0.0:1787827241.369236:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 75 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369244:0:48385:0:(rw26.c:875:ll_write_end()) pos 307200, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369247:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005cc5b8f5[3 0000000007b63142 2 1 00000000746d8de4 0 0 75] 00000080:00200000:0.0:1787827241.369249:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000096e31e84 17ffffc0001001 3:0 ffff9a7885aa81d0 75 lru 00000080:00200000:0.0:1787827241.369252:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000e19d6ba 75: 1< 0 - - > 2< 307200 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369253:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005cc5b8f5 00000080:00200000:0.0:1787827241.369254:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 76. 00000080:00200000:0.0:1787827241.369255:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 76 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369263:0:48385:0:(rw26.c:875:ll_write_end()) pos 311296, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369265:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000094b54764[3 0000000007b63142 2 1 00000000746d8de4 0 0 76] 00000080:00200000:0.0:1787827241.369268:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000233d57c3 17ffffc0001001 3:0 ffff9a7885aa8740 76 lru 00000080:00200000:0.0:1787827241.369271:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000014a90567 76: 1< 0 - - > 2< 311296 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369273:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000094b54764 00000080:00200000:0.0:1787827241.369273:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 77. 00000080:00200000:0.0:1787827241.369274:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 77 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369283:0:48385:0:(rw26.c:875:ll_write_end()) pos 315392, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369285:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000576e6cfb[3 0000000007b63142 2 1 00000000746d8de4 0 0 77] 00000080:00200000:0.0:1787827241.369287:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000995997e1 17ffffc0001001 3:0 ffff9a7885aa8e80 77 lru 00000080:00200000:0.0:1787827241.369290:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004a012313 77: 1< 0 - - > 2< 315392 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369291:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000576e6cfb 00000080:00200000:0.0:1787827241.369292:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 78. 00000080:00200000:0.0:1787827241.369293:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 78 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369301:0:48385:0:(rw26.c:875:ll_write_end()) pos 319488, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369303:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cf154f86[3 0000000007b63142 2 1 00000000746d8de4 0 0 78] 00000080:00200000:0.0:1787827241.369305:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b43a3336 17ffffc0001001 3:0 ffff9a7885aa8488 78 lru 00000080:00200000:0.0:1787827241.369309:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bf68dab7 78: 1< 0 - - > 2< 319488 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369310:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cf154f86 00000080:00200000:0.0:1787827241.369311:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 79. 00000080:00200000:0.0:1787827241.369312:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 79 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369324:0:48385:0:(rw26.c:875:ll_write_end()) pos 323584, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369326:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008d55720f[3 0000000007b63142 2 1 00000000746d8de4 0 0 79] 00000080:00200000:0.0:1787827241.369329:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fd1e3c02 17ffffc0001021 2:0 ffff9a7885aa8bc8 79 lru 00000080:00200000:0.0:1787827241.369332:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002e630c2e 79: 1< 0 - - > 2< 323584 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369334:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008d55720f 00000080:00200000:0.0:1787827241.369335:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 80. 00000080:00200000:0.0:1787827241.369336:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 80 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369344:0:48385:0:(rw26.c:875:ll_write_end()) pos 327680, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369346:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e8f74a55[3 0000000007b63142 2 1 00000000746d8de4 0 0 80] 00000080:00200000:0.0:1787827241.369348:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000065cc3425 17ffffc0001001 3:0 ffff9a7885aa8570 80 lru 00000080:00200000:0.0:1787827241.369351:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f55534f7 80: 1< 0 - - > 2< 327680 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369353:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e8f74a55 00000080:00200000:0.0:1787827241.369354:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 81. 00000080:00200000:0.0:1787827241.369355:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 81 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369370:0:48385:0:(rw26.c:875:ll_write_end()) pos 331776, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369374:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009961cbdc[3 0000000007b63142 2 1 00000000746d8de4 0 0 81] 00000080:00200000:0.0:1787827241.369376:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000173c1d57 17ffffc0001001 3:0 ffff9a7886aac2b8 81 lru 00000080:00200000:0.0:1787827241.369379:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f883656c 81: 1< 0 - - > 2< 331776 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369381:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009961cbdc 00000080:00200000:0.0:1787827241.369382:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 82. 00000080:00200000:0.0:1787827241.369383:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 82 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369391:0:48385:0:(rw26.c:875:ll_write_end()) pos 335872, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369394:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b485afd4[3 0000000007b63142 2 1 00000000746d8de4 0 0 82] 00000080:00200000:0.0:1787827241.369395:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fb6db1e7 17ffffc0001001 3:0 ffff9a7886aac1d0 82 lru 00000080:00200000:0.0:1787827241.369399:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007c8dd934 82: 1< 0 - - > 2< 335872 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369400:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b485afd4 00000080:00200000:0.0:1787827241.369401:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 83. 00000080:00200000:0.0:1787827241.369402:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 83 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369411:0:48385:0:(rw26.c:875:ll_write_end()) pos 339968, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369413:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dc4b4569[3 0000000007b63142 2 1 00000000746d8de4 0 0 83] 00000080:00200000:0.0:1787827241.369416:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fe7e8659 17ffffc0001001 3:0 ffff9a7886aac740 83 lru 00000080:00200000:0.0:1787827241.369419:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003e0c478f 83: 1< 0 - - > 2< 339968 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369421:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dc4b4569 00000080:00200000:0.0:1787827241.369421:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 84. 00000080:00200000:0.0:1787827241.369423:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 84 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369430:0:48385:0:(rw26.c:875:ll_write_end()) pos 344064, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369433:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000274a8aac[3 0000000007b63142 2 1 00000000746d8de4 0 0 84] 00000080:00200000:0.0:1787827241.369434:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001d48db96 17ffffc0001001 3:0 ffff9a7886aace80 84 lru 00000080:00200000:0.0:1787827241.369438:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cca6b48a 84: 1< 0 - - > 2< 344064 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369439:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000274a8aac 00000080:00200000:0.0:1787827241.369440:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 85. 00000080:00200000:0.0:1787827241.369441:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 85 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369449:0:48385:0:(rw26.c:875:ll_write_end()) pos 348160, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369451:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000079ccc089[3 0000000007b63142 2 1 00000000746d8de4 0 0 85] 00000080:00200000:0.0:1787827241.369453:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006293d07e 17ffffc0001001 3:0 ffff9a7886aac488 85 lru 00000080:00200000:0.0:1787827241.369457:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000be83f789 85: 1< 0 - - > 2< 348160 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369458:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000079ccc089 00000080:00200000:0.0:1787827241.369458:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 86. 00000080:00200000:0.0:1787827241.369459:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 86 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369469:0:48385:0:(rw26.c:875:ll_write_end()) pos 352256, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369471:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000090f77647[3 0000000007b63142 2 1 00000000746d8de4 0 0 86] 00000080:00200000:0.0:1787827241.369473:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e4a94c99 17ffffc0001001 3:0 ffff9a7886aacbc8 86 lru 00000080:00200000:0.0:1787827241.369477:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ed98db04 86: 1< 0 - - > 2< 352256 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369478:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000090f77647 00000080:00200000:0.0:1787827241.369478:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 87. 00000080:00200000:0.0:1787827241.369479:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 87 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369487:0:48385:0:(rw26.c:875:ll_write_end()) pos 356352, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369490:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000069124a3c[3 0000000007b63142 2 1 00000000746d8de4 0 0 87] 00000080:00200000:0.0:1787827241.369492:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000703bf485 17ffffc0001001 3:0 ffff9a7886aac570 87 lru 00000080:00200000:0.0:1787827241.369495:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000041028703 87: 1< 0 - - > 2< 356352 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369496:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000069124a3c 00000080:00200000:0.0:1787827241.369497:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 88. 00000080:00200000:0.0:1787827241.369498:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 88 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369526:0:48385:0:(rw26.c:875:ll_write_end()) pos 360448, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369529:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b7a821c3[3 0000000007b63142 2 1 00000000746d8de4 0 0 88] 00000080:00200000:0.0:1787827241.369532:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003c967030 17ffffc0001001 3:0 ffff9a7886aac828 88 lru 00000080:00200000:0.0:1787827241.369536:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000c89d21a 88: 1< 0 - - > 2< 360448 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369537:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b7a821c3 00000080:00200000:0.0:1787827241.369538:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 89. 00000080:00200000:0.0:1787827241.369539:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 89 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369549:0:48385:0:(rw26.c:875:ll_write_end()) pos 364544, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369552:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d8e95094[3 0000000007b63142 2 1 00000000746d8de4 0 0 89] 00000080:00200000:0.0:1787827241.369554:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e0867b31 17ffffc0001001 3:0 ffff9a7886aac000 89 lru 00000080:00200000:0.0:1787827241.369557:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e31e0924 89: 1< 0 - - > 2< 364544 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369559:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d8e95094 00000080:00200000:0.0:1787827241.369560:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 90. 00000080:00200000:0.0:1787827241.369561:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 90 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369569:0:48385:0:(rw26.c:875:ll_write_end()) pos 368640, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369572:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fe48254a[3 0000000007b63142 2 1 00000000746d8de4 0 0 90] 00000080:00200000:0.0:1787827241.369574:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008429d66e 17ffffc0001001 3:0 ffff9a7886aaccb0 90 lru 00000080:00200000:0.0:1787827241.369578:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000009e483c5 90: 1< 0 - - > 2< 368640 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369580:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fe48254a 00000080:00200000:0.0:1787827241.369580:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 91. 00000080:00200000:0.0:1787827241.369582:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 91 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369590:0:48385:0:(rw26.c:875:ll_write_end()) pos 372736, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369593:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cf191ffa[3 0000000007b63142 2 1 00000000746d8de4 0 0 91] 00000080:00200000:0.0:1787827241.369594:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000077a74891 17ffffc0001001 3:0 ffff9a7886aacae0 91 lru 00000080:00200000:0.0:1787827241.369598:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004cd997df 91: 1< 0 - - > 2< 372736 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369599:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cf191ffa 00000080:00200000:0.0:1787827241.369599:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 92. 00000080:00200000:0.0:1787827241.369600:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 92 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369608:0:48385:0:(rw26.c:875:ll_write_end()) pos 376832, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369611:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b518ef0a[3 0000000007b63142 2 1 00000000746d8de4 0 0 92] 00000080:00200000:0.0:1787827241.369612:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ea9453e3 17ffffc0001001 3:0 ffff9a7886aac658 92 lru 00000080:00200000:0.0:1787827241.369616:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000586a9a20 92: 1< 0 - - > 2< 376832 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369617:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b518ef0a 00000080:00200000:0.0:1787827241.369617:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 93. 00000080:00200000:0.0:1787827241.369618:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 93 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369628:0:48385:0:(rw26.c:875:ll_write_end()) pos 380928, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369630:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c608688e[3 0000000007b63142 2 1 00000000746d8de4 0 0 93] 00000080:00200000:0.0:1787827241.369632:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000927236b4 17ffffc0001001 3:0 ffff9a7886aac9f8 93 lru 00000080:00200000:0.0:1787827241.369635:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009ca3850f 93: 1< 0 - - > 2< 380928 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369637:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c608688e 00000080:00200000:0.0:1787827241.369638:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 94. 00000080:00200000:0.0:1787827241.369639:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 94 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369649:0:48385:0:(rw26.c:875:ll_write_end()) pos 385024, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369652:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000070e16961[3 0000000007b63142 2 1 00000000746d8de4 0 0 94] 00000080:00200000:0.0:1787827241.369655:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000060ea9666 17ffffc0001021 2:0 ffff9a7886aac910 94 lru 00000080:00200000:0.0:1787827241.369658:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c2bc7236 94: 1< 0 - - > 2< 385024 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369660:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000070e16961 00000080:00200000:0.0:1787827241.369661:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 95. 00000080:00200000:0.0:1787827241.369662:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 95 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369670:0:48385:0:(rw26.c:875:ll_write_end()) pos 389120, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369673:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007993f515[3 0000000007b63142 2 1 00000000746d8de4 0 0 95] 00000080:00200000:0.0:1787827241.369675:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000067b2c1a1 17ffffc0001001 3:0 ffff9a7886aacd98 95 lru 00000080:00200000:0.0:1787827241.369678:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cdd02a0f 95: 1< 0 - - > 2< 389120 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369680:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007993f515 00000080:00200000:0.0:1787827241.369680:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 96. 00000080:00200000:0.0:1787827241.369681:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 96 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369689:0:48385:0:(rw26.c:875:ll_write_end()) pos 393216, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369692:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000899f32e4[3 0000000007b63142 2 1 00000000746d8de4 0 0 96] 00000080:00200000:0.0:1787827241.369693:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cd7e0942 17ffffc0001001 3:0 ffff9a7886aac3a0 96 lru 00000080:00200000:0.0:1787827241.369697:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a7ef6f07 96: 1< 0 - - > 2< 393216 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369698:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000899f32e4 00000080:00200000:0.0:1787827241.369698:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 97. 00000080:00200000:0.0:1787827241.369700:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 97 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369708:0:48385:0:(rw26.c:875:ll_write_end()) pos 397312, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369711:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f1163f36[3 0000000007b63142 2 1 00000000746d8de4 0 0 97] 00000080:00200000:0.0:1787827241.369713:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a59361bb 17ffffc0001001 3:0 ffff9a7886aac0e8 97 lru 00000080:00200000:0.0:1787827241.369716:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a2dfe5c4 97: 1< 0 - - > 2< 397312 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369717:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f1163f36 00000080:00200000:0.0:1787827241.369718:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 98. 00000080:00200000:0.0:1787827241.369719:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 98 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369733:0:48385:0:(rw26.c:875:ll_write_end()) pos 401408, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369736:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c810dfc6[3 0000000007b63142 2 1 00000000746d8de4 0 0 98] 00000080:00200000:0.0:1787827241.369738:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009da42428 17ffffc0001001 3:0 ffff9a789089cd98 98 lru 00000080:00200000:0.0:1787827241.369741:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000dff33caa 98: 1< 0 - - > 2< 401408 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369742:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c810dfc6 00000080:00200000:0.0:1787827241.369744:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 99. 00000080:00200000:0.0:1787827241.369745:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 99 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369753:0:48385:0:(rw26.c:875:ll_write_end()) pos 405504, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369756:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003a2caa76[3 0000000007b63142 2 1 00000000746d8de4 0 0 99] 00000080:00200000:0.0:1787827241.369758:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f080f462 17ffffc0001001 3:0 ffff9a789089c3a0 99 lru 00000080:00200000:0.0:1787827241.369762:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000093271851 99: 1< 0 - - > 2< 405504 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369763:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003a2caa76 00000080:00200000:0.0:1787827241.369764:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 100. 00000080:00200000:0.0:1787827241.369765:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 100 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369775:0:48385:0:(rw26.c:875:ll_write_end()) pos 409600, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369777:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000025df888f[3 0000000007b63142 2 1 00000000746d8de4 0 0 100] 00000080:00200000:0.0:1787827241.369779:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ccf085f9 17ffffc0001001 3:0 ffff9a789089c0e8 100 lru 00000080:00200000:0.0:1787827241.369782:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000002db86da 100: 1< 0 - - > 2< 409600 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369784:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000025df888f 00000080:00200000:0.0:1787827241.369785:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 101. 00000080:00200000:0.0:1787827241.369786:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 101 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369794:0:48385:0:(rw26.c:875:ll_write_end()) pos 413696, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369797:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003d575a91[3 0000000007b63142 2 1 00000000746d8de4 0 0 101] 00000080:00200000:0.0:1787827241.369799:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000021e56c8e 17ffffc0001001 3:0 ffff9a789089c2b8 101 lru 00000080:00200000:0.0:1787827241.369803:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000652077d1 101: 1< 0 - - > 2< 413696 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369805:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003d575a91 00000080:00200000:0.0:1787827241.369805:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 102. 00000080:00200000:0.0:1787827241.369806:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 102 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369814:0:48385:0:(rw26.c:875:ll_write_end()) pos 417792, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369816:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006f2b18d1[3 0000000007b63142 2 1 00000000746d8de4 0 0 102] 00000080:00200000:0.0:1787827241.369818:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000085e27235 17ffffc0001001 3:0 ffff9a789089c1d0 102 lru 00000080:00200000:0.0:1787827241.369821:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000ea71e00 102: 1< 0 - - > 2< 417792 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369822:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006f2b18d1 00000080:00200000:0.0:1787827241.369823:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 103. 00000080:00200000:0.0:1787827241.369824:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 103 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369857:0:48385:0:(rw26.c:875:ll_write_end()) pos 421888, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369863:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f68fd160[3 0000000007b63142 2 1 00000000746d8de4 0 0 103] 00000080:00200000:0.0:1787827241.369868:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003904688c 17ffffc0001001 3:0 ffff9a789089c740 103 lru 00000080:00200000:0.0:1787827241.369874:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004e5faca9 103: 1< 0 - - > 2< 421888 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369878:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f68fd160 00000080:00200000:0.0:1787827241.369879:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 104. 00000080:00200000:0.0:1787827241.369881:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 104 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369895:0:48385:0:(rw26.c:875:ll_write_end()) pos 425984, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369898:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ba314061[3 0000000007b63142 2 1 00000000746d8de4 0 0 104] 00000080:00200000:0.0:1787827241.369900:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000045071a2a 17ffffc0001001 3:0 ffff9a789089ce80 104 lru 00000080:00200000:0.0:1787827241.369904:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ef48ebc4 104: 1< 0 - - > 2< 425984 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369906:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ba314061 00000080:00200000:0.0:1787827241.369907:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 105. 00000080:00200000:0.0:1787827241.369908:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 105 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.369917:0:48385:0:(rw26.c:875:ll_write_end()) pos 430080, len 4096, copied 4096 00000080:00200000:0.0:1787827241.369919:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007949b8bd[3 0000000007b63142 2 1 00000000746d8de4 0 0 105] 00000080:00200000:0.0:1787827241.369921:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006dfd84f7 17ffffc0001001 3:0 ffff9a789089c488 105 lru 00000080:00200000:0.0:1787827241.369925:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006695e8e6 105: 1< 0 - - > 2< 430080 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369927:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007949b8bd 00000080:00200000:0.0:1787827241.369928:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 106. 00000080:00200000:0.0:1787827241.369929:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 106 of 0 to 405 bytes 00000080:00200000:0.0:1787827241.369940:0:48385:0:(rw26.c:875:ll_write_end()) pos 434176, len 405, copied 405 00000080:00200000:0.0:1787827241.369943:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c8a5158c[3 0000000007b63142 2 1 00000000746d8de4 0 0 106] 00000080:00200000:0.0:1787827241.369945:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002a4e4511 17ffffc0001001 3:0 ffff9a789089cbc8 106 lru 00000080:00200000:0.0:1787827241.369948:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007fa44ca3 106: 1< 0 - - > 2< 434176 0 0 0x0 0x0 | 0000000023a82262 000000007dedc6c9 > 3< 0 0 > 4< 0 0 8 2572288 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.369950:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c8a5158c 00000080:00200000:0.0:1787827241.369950:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 107. 00000080:00200000:0.0:1787827241.369953:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 107, from 0, to 405 prio 0 00000008:00010000:0.0:1787827241.369974:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.369989:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.369997:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370002:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370007:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370012:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370017:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370022:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370027:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370032:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370036:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370042:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370048:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370053:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370057:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370067:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.370071:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370076:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370081:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370086:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370097:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370102:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370107:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370112:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370117:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370121:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370126:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370131:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370136:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370142:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370148:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370154:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.370157:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370162:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370167:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370172:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370177:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370182:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370190:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370196:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370201:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370206:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370210:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370215:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370220:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370225:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370230:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370238:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.370241:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370246:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370251:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370256:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370261:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370266:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370271:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370275:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370284:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370288:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370293:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370298:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370303:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370308:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370313:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370320:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.370322:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370331:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370336:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370340:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370345:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370350:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370355:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370360:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370365:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370370:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370375:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370380:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370385:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370389:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370394:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370401:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.370404:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370409:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370414:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370422:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370427:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370432:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370436:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370441:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370446:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370451:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370456:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370461:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370466:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370471:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370475:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370482:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.370485:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370490:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370495:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370499:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370520:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370529:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370535:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370540:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370544:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370549:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370554:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370559:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370564:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370568:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370575:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370583:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.370587:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.370592:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008760f88c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.370598:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc92f8, count 2, dirtied 2 00000080:00200000:0.0:1787827241.370601:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 107 pages 434581 bytes, tot: 434581 00000080:00200000:0.0:1787827241.370603:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) sanity-sec.sh: write bytes 0, result: 434581 00000020:00010000:0.0:1787827241.370614:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.370616:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.370618:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f9b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.370621:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.370628:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.370636:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.370638:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f9b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.370642:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 434581 bytes 00000080:00200000:0.0:1787827241.370647:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 106] 00000080:00200000:0.0:1787827241.370652:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x45:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.370655:0:48385:0:(file.c:2240:ll_file_io_generic()) sanity-sec.sh: 2 io complete with rc: 0, result: 434581, restart: 0 00000080:00200000:0.0:1787827241.370660:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x45:0x0] updating i_size 434581 i_blocks 0 00000080:00200000:0.0:1787827241.370663:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 434581 00000080:00200008:0.0:1787827241.370682:0:48385:0:(file.c:479:ll_file_release()) START file sanity-sec.sh:[0x240001b71:0x45:0x0](ffff9a789dfc9148), flags 100001 00000002:00100000:0.0:1787827241.370708:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a667f840 x1874671476636544/t17179869509(17179869509) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.370716:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.370720:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a667fc00 x1874671476637184/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.370731:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a667fc00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637184:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.370765:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4780 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.371559:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a667fc00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637184:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.371572:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a667f840 x1874671476636544/t17179869509(17179869509) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.371607:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.371616:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.371626:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.371634:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.371640:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.371643:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.371645:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.371650:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.371652:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.371658:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.371666:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.371669:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.371673:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.371675:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.371677:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.371679:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=sanity-sec.sh, flags=8 00000080:00200008:0.0:1787827241.371684:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-sec.sh:[0x240001b71:0x45:0x0](ffff9a789dfc9148), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.371687:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x45:0x0](ffff9a789dfc9148),name=sanity-sec.sh 00000002:00010000:0.0:1787827241.371694:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x45:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.371699:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18600/0xebdcd48a4298edf7 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca15 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.371705:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18600/0xebdcd48a4298edf7 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca15 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.371709:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.371712:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x45:0x0](ffff9a789dfc9148) 00000080:00010000:0.0:1787827241.371713:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x45:0x0](ffff9a789dfc9148) for lock 0xebdcd48a4298edf7 00000080:00010000:0.0:1787827241.371715:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298edf7 from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.371717:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18600/0xebdcd48a4298edf7 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dca15 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.371727:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18600/0xebdcd48a4298edf7 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca15 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.371737:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x45:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.371742:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x45:0x0] 00000008:00010000:0.0:1787827241.371749:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.371751:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f9b0, osc lock ffff9a7886bc5110, flags 0x400001002 00010000:00010000:0.0:1787827241.371758:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.371765:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.371772:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.371774:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.371780:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000030f195d3/0xebdcd48a4298ee05 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xa3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->438271) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca2a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.371786:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.371790:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x45:0x0] updating i_size 434581 i_blocks 0 00000020:00010000:0.0:1787827241.371792:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.371794:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.371796:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f9b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.371798:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.371800:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f9b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.371805:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x45:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.371858:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.371868:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.371875:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.371884:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.371889:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.371892:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.371894:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.371897:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.371900:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.371905:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.371912:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.371914:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.371917:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.371919:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.371920:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.371927:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-selinux.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.371931:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-selinux.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.371938:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-selinux.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.371959:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.371965:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.371971:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.371979:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637312:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.372009:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4780 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.372903:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637312:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.372911:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372918:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372923:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372928:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372933:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372936:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a19800 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.372940:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372952:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372957:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372968:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.372971:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.372976:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee0c lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.372986:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a65303c0 x1874671476637312/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.372994:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.373005:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.373031:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.373037:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.373042:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.373048:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.373054:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.373057:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.373059:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.373063:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.373066:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.373071:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.373078:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.373081:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.373083:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.373085:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.373087:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.373092:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity-selinux.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:0.0:1787827241.373094:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-selinux.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.373099:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-selinux.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.373121:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.373126:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.373132:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.373140:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637440:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.373167:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4780 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.374132:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637440:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.374139:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374145:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374149:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374154:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374158:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374161:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a19e00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.374164:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374167:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374171:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374174:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.374177:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374181:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee13 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.374189:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a65303c0 x1874671476637440/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.374200:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.374208:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.374251:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.374257:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.374262:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.374268:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.374273:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.374276:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.374278:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.374281:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:0.0:1787827241.374283:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.374288:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.374294:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.374297:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.374299:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.374302:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.374303:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.374310:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.374314:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=sanity-selinux.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78a6580600, open_flags 80c1, mode 81ed opened 0 00000080:00200000:0.0:1787827241.374318:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity-selinux.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:0.0:1787827241.374330:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity-selinux.sh,[0x240001b71:0x46:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:0.0:1787827241.374349:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.374354:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827241.374360:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827241.374364:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65303c0 x1874671476637696/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827241.374370:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.374378:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637696:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.374453:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4780 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.375678:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65303c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637696:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.375689:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375711:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375717:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375726:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375734:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375741:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375746:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375751:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.375757:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a65303c0 x1874671476637696/t17179869511(17179869511) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827241.375766:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a65303c0 x1874671476637696/t17179869511(17179869511) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827241.375779:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827241.375791:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375800:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/70 (0 0) 00000080:00200000:0.0:1787827241.375805:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827241.375813:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=sanity-selinux.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:0.0:1787827241.375855:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x46:0x0](ffff9a789dfcba88) 00000080:00200000:0.0:1787827241.375909:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x46:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787827241.375913:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x46:0x0](ffff9a789dfcba88) 00000080:00010000:0.0:1787827241.375915:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x46:0x0](ffff9a789dfcba88) for lock 0xebdcd48a4298ee1a 00000080:00010000:0.0:1787827241.375921:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x46:0x0](ffff9a789dfcba88) 00000080:00200008:0.0:1787827241.375927:0:48385:0:(file.c:1090:ll_file_open()) START file sanity-selinux.sh:[0x240001b71:0x46:0x0](ffff9a789dfcba88), flags 100301 00000002:00100000:0.0:1787827241.375941:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a65303c0 x1874671476637696/t17179869511(17179869511) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827241.375968:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee1a from it ffff9a7887be0720 00010000:00010000:0.0:1787827241.375971:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.375977:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.375988:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x46:0x0](ffff9a789dfcba88), inode mode 81ed mask 40 00000080:00200008:0.0:1787827241.375997:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-selinux.sh:[0x240001b71:0x46:0x0](ffff9a789dfcba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.376000:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x46:0x0](ffff9a789dfcba88),name=sanity-selinux.sh 00000002:00010000:0.0:1787827241.376008:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x46:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.376013:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.376020:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.376025:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.376028:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x46:0x0](ffff9a789dfcba88) 00000080:00010000:0.0:1787827241.376029:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x46:0x0](ffff9a789dfcba88) for lock 0xebdcd48a4298ee1a 00000080:00010000:0.0:1787827241.376032:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee1a from it ffffaa5487e7bcd8 00010000:00010000:0.0:1787827241.376035:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.376040:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.376047:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x46:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.376051:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x46:0x0] 00000008:00010000:0.0:1787827241.376058:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.376060:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f5b0, osc lock ffff9a7886bc5110, flags 0x400001002 00010000:00010000:0.0:1787827241.376080:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.376087:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.376112:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.376135:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476637824:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.376915:0:10934:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.376924:0:10934:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.376933:0:10934:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.376941:0:10934:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.376949:0:10934:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 5/2,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.376956:0:10934:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1787827241.376966:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.376973:0:10934:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.376978:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.376984:0:10934:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1787827241.376994:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476637824:192.168.203.140@tcp:101:cp.0 00000080:00200000:2.0:1787827241.376995:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x46:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.377001:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.377004:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.377006:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f5b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.377009:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.377012:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f5b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.377020:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x46:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.377051:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity-selinux.sh:[0x240001b71:0x46:0x0], ppos: 0, count: 26716 00000080:00200000:2.0:1787827241.377057:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity-selinux.sh: write ppos: 0, bytes: 26716 00000080:00200000:2.0:1787827241.377061:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity-selinux.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.377067:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000a47616c0[0x110, 1, [0x240001b71:0x46:0x0] hash] 00000020:00200000:2.0:1787827241.377068:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 26716) 0 0 00000080:00200000:2.0:1787827241.377070:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x46:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.377076:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 6] 00020000:00200000:2.0:1787827241.377081:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 26716) 26716, 26716 00020000:00200000:2.0:1787827241.377084:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.377092:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 6] 00000008:00010000:2.0:1787827241.377101:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.377103:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c4b0, osc lock ffff9a7884063e78, flags 0x0 00010000:00010000:2.0:1787827241.377109:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/164 (0 28671) 00010000:00010000:2.0:1787827241.377128:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.377136:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->28671] (req 0->28671) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.377142:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->28671] (req 0->28671) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.377152:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637952:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.377186:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.378638:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.378733:0:48371:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787827241.378753:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786048 00000100:00100000:1.0:1787827241.378766:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786048 00000100:00100000:1.0:1787827241.378825:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 293 00000100:00100000:1.0:1787827241.378880:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6438450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488786048:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.378902:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000bf2b4edd/0xebdcd48a4298ee21 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dca7e expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.378976:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.378985:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6438450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488786048:12345-192.168.203.140@tcp:104: Request processed in 127us (352us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.378999:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 293 00000100:00100000:1.0:1787827241.379210:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476638080:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.379950:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476637952:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.379979:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->28671] (req 0->28671) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.379989:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->28671] (req 0->28671) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.379997:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->28671] (req 0->28671) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.380202:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:2.0:1787827241.380243:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786176 00000100:00100000:2.0:1787827241.380248:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786176 00000100:00100000:2.0:1787827241.380277:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 294 00000100:00100000:2.0:1787827241.380283:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6530000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488786176:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.380290:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->28671] (req 0->28671) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.380298:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->28671] (req 0->28671) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.380306:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.380314:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.380321:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.380328:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.380345:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890907e00) 00000100:00100000:2.0:1787827241.380375:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6530000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488786176:12345-192.168.203.140@tcp:105: Request processed in 91us (173us total) trans 0 rc 0/0 00010000:00010000:0.0:1787827241.380378:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.380393:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 294 00010000:00010000:0.0:1787827241.380397:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.380423:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.380432:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,2 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.380441:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.380447:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.380474:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) sanity-selinux.sh: write [0, 26716) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.380491:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.380547:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827241.380557:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d9f8e3ef[3 00000000eb4d4053 2 1 00000000746d8de4 0 0 0] 00000080:00200000:0.0:1787827241.380560:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002cd291e0 17ffffc0001001 3:0 ffff9a789089c570 0 lru 00000080:00200000:0.0:1787827241.380567:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006e02c001 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 000000004720a2fa > 3< 0 0 > 4< 0 0 8 1748992 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.380569:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d9f8e3ef 00000080:00200000:0.0:1787827241.380570:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.380572:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.380588:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:0.0:1787827241.380592:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000045a397dc[3 00000000eb4d4053 2 1 00000000746d8de4 0 0 1] 00000080:00200000:0.0:1787827241.380594:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e1ad0b48 17ffffc0001001 3:0 ffff9a789089c828 1 lru 00000080:00200000:0.0:1787827241.380598:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003b579256 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000043fddcb7 000000004720a2fa > 3< 0 0 > 4< 0 0 8 1748992 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.380600:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000045a397dc 00000080:00200000:0.0:1787827241.380601:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:0.0:1787827241.380602:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.380619:0:48385:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:0.0:1787827241.380622:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003997a936[3 00000000eb4d4053 2 1 00000000746d8de4 0 0 2] 00000080:00200000:0.0:1787827241.380625:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ab3ea769 17ffffc0001021 2:0 ffff9a789089c000 2 lru 00000080:00200000:0.0:1787827241.380629:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000179f41f1 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000043fddcb7 000000004720a2fa > 3< 0 0 > 4< 0 0 8 1748992 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.380630:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003997a936 00000080:00200000:0.0:1787827241.380631:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:0.0:1787827241.380633:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.380642:0:48385:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:0.0:1787827241.380644:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b40f7bae[3 00000000eb4d4053 2 1 00000000746d8de4 0 0 3] 00000080:00200000:0.0:1787827241.380646:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000004b04a88 17ffffc0001001 3:0 ffff9a789089ccb0 3 lru 00000080:00200000:0.0:1787827241.380650:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c035c7ec 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000043fddcb7 000000004720a2fa > 3< 0 0 > 4< 0 0 8 1748992 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.380651:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b40f7bae 00000080:00200000:0.0:1787827241.380652:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:0.0:1787827241.380653:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.380662:0:48385:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:0.0:1787827241.380664:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000046cef24a[3 00000000eb4d4053 2 1 00000000746d8de4 0 0 4] 00000080:00200000:0.0:1787827241.380666:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000096e8081a 17ffffc0001001 3:0 ffff9a789089cae0 4 lru 00000080:00200000:0.0:1787827241.380670:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c091b07a 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000043fddcb7 000000004720a2fa > 3< 0 0 > 4< 0 0 8 1748992 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.380672:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000046cef24a 00000080:00200000:0.0:1787827241.380672:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:0.0:1787827241.380674:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.380681:0:48385:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:0.0:1787827241.380684:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000059165e86[3 00000000eb4d4053 2 1 00000000746d8de4 0 0 5] 00000080:00200000:0.0:1787827241.380686:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002cf1d6f8 17ffffc0001001 3:0 ffff9a789089c658 5 lru 00000080:00200000:0.0:1787827241.380689:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000068ce141f 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000043fddcb7 000000004720a2fa > 3< 0 0 > 4< 0 0 8 1748992 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.380690:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000059165e86 00000080:00200000:0.0:1787827241.380691:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:0.0:1787827241.380692:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 2140 bytes 00000080:00200000:0.0:1787827241.380710:0:48385:0:(rw26.c:875:ll_write_end()) pos 24576, len 2140, copied 2140 00000080:00200000:0.0:1787827241.380713:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000057714ca6[3 00000000eb4d4053 2 1 00000000746d8de4 0 0 6] 00000080:00200000:0.0:1787827241.380715:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000080a7a346 17ffffc0001001 3:0 ffff9a789089c9f8 6 lru 00000080:00200000:0.0:1787827241.380719:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004cac47bd 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000043fddcb7 000000004720a2fa > 3< 0 0 > 4< 0 0 8 1748992 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.380721:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000057714ca6 00000080:00200000:0.0:1787827241.380722:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:0.0:1787827241.380724:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 7, from 0, to 2140 prio 0 00000008:00010000:0.0:1787827241.380746:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000b97a8cbb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.380764:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000b97a8cbb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.380771:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000b97a8cbb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.380776:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000b97a8cbb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.380781:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000b97a8cbb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.380786:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000b97a8cbb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.380791:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000b97a8cbb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.380800:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcbc38, count 7, dirtied 7 00000080:00200000:0.0:1787827241.380803:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 7 pages 26716 bytes, tot: 26716 00000080:00200000:0.0:1787827241.380805:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) sanity-selinux.sh: write bytes 0, result: 26716 00000020:00010000:0.0:1787827241.380819:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.380822:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.380826:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c4b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.380865:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.380876:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.380888:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.380891:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c4b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.380904:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 26716 bytes 00000080:00200000:0.0:1787827241.380913:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 6] 00000080:00200000:0.0:1787827241.380920:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x46:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.380924:0:48385:0:(file.c:2240:ll_file_io_generic()) sanity-selinux.sh: 2 io complete with rc: 0, result: 26716, restart: 0 00000080:00200000:0.0:1787827241.380931:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x46:0x0] updating i_size 26716 i_blocks 0 00000080:00200000:0.0:1787827241.380936:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 26716 00000080:00200008:0.0:1787827241.380980:0:48385:0:(file.c:479:ll_file_release()) START file sanity-selinux.sh:[0x240001b71:0x46:0x0](ffff9a789dfcba88), flags 100001 00000100:00100000:1.0:1787827241.380992:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476638080:192.168.203.140@tcp:103:ldlm_bl.0 00000002:00100000:0.0:1787827241.381023:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a65303c0 x1874671476637696/t17179869511(17179869511) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.381033:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.381038:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6531a40 x1874671476638336/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.381050:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531a40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638336:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.381085:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.381928:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531a40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638336:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.381946:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a65303c0 x1874671476637696/t17179869511(17179869511) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.382001:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.382016:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.382031:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.382040:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.382046:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.382050:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.382054:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.382059:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.382061:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.382067:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.382075:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.382080:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.382084:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.382087:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.382089:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.382091:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=sanity-selinux.sh, flags=8 00000080:00200008:0.0:1787827241.382098:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity-selinux.sh:[0x240001b71:0x46:0x0](ffff9a789dfcba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.382101:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x46:0x0](ffff9a789dfcba88),name=sanity-selinux.sh 00000002:00010000:0.0:1787827241.382110:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x46:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.382118:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.382124:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.382128:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.382130:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x46:0x0](ffff9a789dfcba88) 00000080:00010000:0.0:1787827241.382132:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x46:0x0](ffff9a789dfcba88) for lock 0xebdcd48a4298ee1a 00000080:00010000:0.0:1787827241.382134:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee1a from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.382136:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.382141:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.382154:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x46:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.382161:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x46:0x0] 00000008:00010000:0.0:1787827241.382171:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.382173:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2fcb0, osc lock ffff9a7886bc4ea0, flags 0x400001002 00010000:00010000:0.0:1787827241.382183:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.382190:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.382198:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.382201:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.382207:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e9b457f3/0xebdcd48a4298ee28 lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dca85 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.382213:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.382216:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x46:0x0] updating i_size 26716 i_blocks 0 00000020:00010000:0.0:1787827241.382219:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.382221:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.382223:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2fcb0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.382224:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.382226:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2fcb0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.382231:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x46:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.382282:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.382287:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.382292:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.382298:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.382303:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.382306:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.382308:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.382311:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.382314:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.382329:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.382338:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.382343:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.382347:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.382350:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.382353:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.382365:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.382369:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.382378:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.382405:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.382419:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.382428:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.382439:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638464:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.382469:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.383470:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638464:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.383482:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383489:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383495:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383500:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383518:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383523:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a19e00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.383527:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383532:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383537:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383541:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.383547:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383552:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19e00/0xebdcd48a4298ee2f lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.383564:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6530780 x1874671476638464/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.383574:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.383588:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.383616:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.383623:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.383629:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.383636:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.383641:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.383644:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.383646:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.383649:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.383651:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.383657:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.383663:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.383666:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.383669:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.383672:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.383673:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.383679:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sanity.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:0.0:1787827241.383682:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.383687:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.383704:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383713:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.383718:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.383728:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638592:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.383761:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.384658:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638592:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.384668:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384675:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384680:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384685:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384689:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384692:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a19800 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.384695:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384699:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384704:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384708:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.384711:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384714:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19800/0xebdcd48a4298ee36 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.384724:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6530780 x1874671476638592/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.384731:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.384740:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.384782:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.384788:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.384793:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.384800:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.384806:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.384809:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.384810:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.384814:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:0.0:1787827241.384816:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.384826:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.384861:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.384867:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.384871:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.384875:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.384878:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.384889:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.384894:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=sanity.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78a6581b00, open_flags 80c1, mode 81ed opened 0 00000080:00200000:0.0:1787827241.384908:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sanity.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:0.0:1787827241.384923:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sanity.sh,[0x240001b71:0x47:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:0.0:1787827241.384948:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.384954:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827241.384960:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827241.384965:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6530780 x1874671476638848/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827241.384971:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.384980:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638848:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.385083:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.386281:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476638848:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.386290:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386301:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386307:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386315:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386321:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386327:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386332:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386337:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.386343:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6530780 x1874671476638848/t17179869513(17179869513) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827241.386352:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6530780 x1874671476638848/t17179869513(17179869513) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827241.386361:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827241.386368:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386374:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/71 (0 0) 00000080:00200000:0.0:1787827241.386378:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827241.386384:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=sanity.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:0.0:1787827241.386406:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x47:0x0](ffff9a789dfccb08) 00000080:00200000:0.0:1787827241.386468:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x47:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787827241.386472:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x47:0x0](ffff9a789dfccb08) 00000080:00010000:0.0:1787827241.386475:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x47:0x0](ffff9a789dfccb08) for lock 0xebdcd48a4298ee3d 00000080:00010000:0.0:1787827241.386481:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x47:0x0](ffff9a789dfccb08) 00000080:00200008:0.0:1787827241.386488:0:48385:0:(file.c:1090:ll_file_open()) START file sanity.sh:[0x240001b71:0x47:0x0](ffff9a789dfccb08), flags 100301 00000002:00100000:0.0:1787827241.386500:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6530780 x1874671476638848/t17179869513(17179869513) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827241.386543:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee3d from it ffff9a7887be0720 00010000:00010000:0.0:1787827241.386546:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386553:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.386564:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x47:0x0](ffff9a789dfccb08), inode mode 81ed mask 40 00000080:00200008:0.0:1787827241.386573:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity.sh:[0x240001b71:0x47:0x0](ffff9a789dfccb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.386577:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x47:0x0](ffff9a789dfccb08),name=sanity.sh 00000002:00010000:0.0:1787827241.386585:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x47:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.386590:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.386597:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.386601:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.386604:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x47:0x0](ffff9a789dfccb08) 00000080:00010000:0.0:1787827241.386605:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x47:0x0](ffff9a789dfccb08) for lock 0xebdcd48a4298ee3d 00000080:00010000:0.0:1787827241.386608:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee3d from it ffffaa5487e7bcd8 00010000:00010000:0.0:1787827241.386610:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.386616:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.386625:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x47:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.386631:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x47:0x0] 00000008:00010000:0.0:1787827241.386639:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.386642:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f930, osc lock ffff9a7886bc4ea0, flags 0x400001002 00010000:00010000:0.0:1787827241.386674:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.386681:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.386713:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.388115:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.388136:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.388153:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.388165:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.388180:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.388192:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1787827241.388211:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.388225:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.388234:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.388244:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1787827241.388262:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65312c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476638976:192.168.203.140@tcp:101:cp.0 00000080:00200000:2.0:1787827241.388266:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x47:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.388275:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.388280:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.388284:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f930 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.388289:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.388294:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f930 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.388308:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x47:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.390619:0:48385:0:(file.c:2652:do_file_write_iter()) START file sanity.sh:[0x240001b71:0x47:0x0], ppos: 0, count: 1200870 00000080:00200000:2.0:1787827241.390632:0:48385:0:(file.c:2090:ll_file_io_generic()) sanity.sh: write ppos: 0, bytes: 1200870 00000080:00200000:2.0:1787827241.390637:0:48385:0:(file.c:1848:ll_io_set_mirror()) sanity.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.390644:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@0000000066873144[0x110, 1, [0x240001b71:0x47:0x0] hash] 00000020:00200000:2.0:1787827241.390646:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 1200870) 0 0 00000080:00200000:2.0:1787827241.390650:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x47:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.390664:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 293] 00020000:00200000:2.0:1787827241.390673:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 1200870) 1200870, 1200870 00020000:00200000:2.0:1787827241.390676:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.390692:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 293] 00000008:00010000:2.0:1787827241.390706:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.390710:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3ccb0, osc lock ffff9a7884063ad0, flags 0x0 00010000:00010000:2.0:1787827241.390725:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419264/164 (0 1204223) 00010000:00010000:2.0:1787827241.390764:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.390773:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->1204223] (req 0->1204223) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.390782:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->1204223] (req 0->1204223) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.390804:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639104:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.390901:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.392025:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.392072:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786304 00000100:00100000:1.0:1787827241.392080:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786304 00000100:00100000:1.0:1787827241.392126:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 295 00000100:00100000:1.0:1787827241.392136:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a65312c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488786304:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.392152:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.392218:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.392226:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a65312c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488786304:12345-192.168.203.140@tcp:104: Request processed in 92us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.392236:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 295 00010000:00010000:2.0:1787827241.392246:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827241.392254:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.392263:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890a19a00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:2.0:1787827241.392269:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.392298:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x47:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.392309:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x47:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827241.392345:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.392355:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.392386:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.392397:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000007d0609c1/0xebdcd48a4298ee44 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcad9 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.392421:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.392425:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.392429:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.393032:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639104:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.393051:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->1204223] (req 0->1204223) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393060:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->1204223] (req 0->1204223) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393067:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->1204223] (req 0->1204223) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.393371:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:2.0:1787827241.393411:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786432 00000100:00100000:2.0:1787827241.393418:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786432 00000100:00100000:2.0:1787827241.393457:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 296 00000100:00100000:2.0:1787827241.393467:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a65321c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488786432:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.393477:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->1204223] (req 0->1204223) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393489:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->1204223] (req 0->1204223) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393500:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393526:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393538:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393549:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.393575:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7890906000) 00010000:00010000:0.0:1787827241.393601:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.393617:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a65321c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488786432:12345-192.168.203.140@tcp:105: Request processed in 150us (243us total) trans 0 rc 0/0 00010000:00010000:0.0:1787827241.393618:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.393635:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 296 00010000:00010000:0.0:1787827241.393645:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.393661:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,2 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.393674:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.393681:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.393708:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) sanity.sh: write [0, 1200870) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.393722:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.393758:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827241.393767:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003c2284d0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 0] 00000080:00200000:0.0:1787827241.393771:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c396bc10 17ffffc0001001 3:0 ffff9a789089c910 0 lru 00000080:00200000:0.0:1787827241.393780:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fa9bf9dd 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.393783:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003c2284d0 00000080:00200000:0.0:1787827241.393785:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.393789:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.393823:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:0.0:1787827241.393856:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000023fe5334[3 00000000bdb38050 2 1 00000000746d8de4 0 0 1] 00000080:00200000:0.0:1787827241.393861:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a158ac92 17ffffc0001001 3:0 ffff9a7886944cb0 1 lru 00000080:00200000:0.0:1787827241.393869:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ed318f36 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.393872:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000023fe5334 00000080:00200000:0.0:1787827241.393874:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:0.0:1787827241.393876:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.393894:0:48385:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:0.0:1787827241.393899:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e30c3b50[3 00000000bdb38050 2 1 00000000746d8de4 0 0 2] 00000080:00200000:0.0:1787827241.393902:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006770a1f3 17ffffc0001001 3:0 ffff9a7886944ae0 2 lru 00000080:00200000:0.0:1787827241.393909:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c8715402 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.393912:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e30c3b50 00000080:00200000:0.0:1787827241.393913:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:0.0:1787827241.393915:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.393927:0:48385:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:0.0:1787827241.393931:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008479b695[3 00000000bdb38050 2 1 00000000746d8de4 0 0 3] 00000080:00200000:0.0:1787827241.393935:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d998a3a6 17ffffc0001001 3:0 ffff9a7886944658 3 lru 00000080:00200000:0.0:1787827241.393941:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003ba92ec8 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.393944:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008479b695 00000080:00200000:0.0:1787827241.393945:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:0.0:1787827241.393947:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.393958:0:48385:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:0.0:1787827241.393961:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c33d6050[3 00000000bdb38050 2 1 00000000746d8de4 0 0 4] 00000080:00200000:0.0:1787827241.393963:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008834b2be 17ffffc0001001 3:0 ffff9a78869449f8 4 lru 00000080:00200000:0.0:1787827241.393967:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000202203e4 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.393969:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c33d6050 00000080:00200000:0.0:1787827241.393970:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:0.0:1787827241.393971:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.393983:0:48385:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:0.0:1787827241.393987:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d7d3556f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 5] 00000080:00200000:0.0:1787827241.393991:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f70bed2c 17ffffc0001001 3:0 ffff9a7886944910 5 lru 00000080:00200000:0.0:1787827241.393997:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000012a6b712 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.393999:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d7d3556f 00000080:00200000:0.0:1787827241.394001:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:0.0:1787827241.394003:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394020:0:48385:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394026:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d511096d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 6] 00000080:00200000:0.0:1787827241.394029:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002f3d0015 17ffffc0001001 3:0 ffff9a7886944d98 6 lru 00000080:00200000:0.0:1787827241.394035:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000449a5d08 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394038:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d511096d 00000080:00200000:0.0:1787827241.394039:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:0.0:1787827241.394041:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394060:0:48385:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394066:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007da3ae4b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 7] 00000080:00200000:0.0:1787827241.394069:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f5073655 17ffffc0001001 3:0 ffff9a78869443a0 7 lru 00000080:00200000:0.0:1787827241.394076:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002f4330b8 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394078:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007da3ae4b 00000080:00200000:0.0:1787827241.394079:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:0.0:1787827241.394081:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394096:0:48385:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394102:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000020e24453[3 00000000bdb38050 2 1 00000000746d8de4 0 0 8] 00000080:00200000:0.0:1787827241.394105:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ccfd91cb 17ffffc0001001 3:0 ffff9a78869440e8 8 lru 00000080:00200000:0.0:1787827241.394112:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ad5d5cfc 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394114:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000020e24453 00000080:00200000:0.0:1787827241.394116:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:0.0:1787827241.394118:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394133:0:48385:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394138:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001c87dcb0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 9] 00000080:00200000:0.0:1787827241.394142:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b5fdb23c 17ffffc0001001 3:0 ffff9a78869442b8 9 lru 00000080:00200000:0.0:1787827241.394148:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000049951335 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394151:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001c87dcb0 00000080:00200000:0.0:1787827241.394153:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:0.0:1787827241.394155:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394174:0:48385:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394179:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006c3eebf8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 10] 00000080:00200000:0.0:1787827241.394183:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c16902f4 17ffffc0001021 2:0 ffff9a78869441d0 10 lru 00000080:00200000:0.0:1787827241.394189:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fb39e383 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394192:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006c3eebf8 00000080:00200000:0.0:1787827241.394194:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:0.0:1787827241.394196:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394209:0:48385:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394214:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e3342f04[3 00000000bdb38050 2 1 00000000746d8de4 0 0 11] 00000080:00200000:0.0:1787827241.394218:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000097ade050 17ffffc0001001 3:0 ffff9a7886944740 11 lru 00000080:00200000:0.0:1787827241.394225:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a3a899fb 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394228:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e3342f04 00000080:00200000:0.0:1787827241.394229:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:0.0:1787827241.394231:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394244:0:48385:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394250:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c15cfb8e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 12] 00000080:00200000:0.0:1787827241.394253:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000e8037c9 17ffffc0001001 3:0 ffff9a7886944e80 12 lru 00000080:00200000:0.0:1787827241.394260:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d1544669 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000100:00100000:1.0:1787827241.394264:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476639232:192.168.203.140@tcp:103:ldlm_bl.0 00000080:00200000:0.0:1787827241.394265:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c15cfb8e 00000080:00200000:0.0:1787827241.394267:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:0.0:1787827241.394271:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394284:0:48385:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394289:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005f43ac17[3 00000000bdb38050 2 1 00000000746d8de4 0 0 13] 00000080:00200000:0.0:1787827241.394293:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000cced031 17ffffc0001001 3:0 ffff9a7886944488 13 lru 00000080:00200000:0.0:1787827241.394297:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001f745a9f 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394299:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005f43ac17 00000080:00200000:0.0:1787827241.394300:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:0.0:1787827241.394301:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394314:0:48385:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394316:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000003fcdc90[3 00000000bdb38050 2 1 00000000746d8de4 0 0 14] 00000080:00200000:0.0:1787827241.394318:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000022ed08ec 17ffffc0001001 3:0 ffff9a7886944bc8 14 lru 00000080:00200000:0.0:1787827241.394322:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002a6ff248 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394324:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000003fcdc90 00000080:00200000:0.0:1787827241.394325:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:0.0:1787827241.394326:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394334:0:48385:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394336:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000038e61c9b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 15] 00000080:00200000:0.0:1787827241.394338:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000271d3218 17ffffc0001001 3:0 ffff9a7886944570 15 lru 00000080:00200000:0.0:1787827241.394342:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007342732f 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394343:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000038e61c9b 00000080:00200000:0.0:1787827241.394344:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:0.0:1787827241.394345:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394354:0:48385:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394356:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d9cdfcd2[3 00000000bdb38050 2 1 00000000746d8de4 0 0 16] 00000080:00200000:0.0:1787827241.394358:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000022117a6e 17ffffc0001001 3:0 ffff9a7886944828 16 lru 00000080:00200000:0.0:1787827241.394361:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a33da1a3 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394363:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d9cdfcd2 00000080:00200000:0.0:1787827241.394363:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000080:00200000:0.0:1787827241.394364:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394373:0:48385:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394375:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001fec64a0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 17] 00000080:00200000:0.0:1787827241.394377:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a560937b 17ffffc0001001 3:0 ffff9a7886944000 17 lru 00000080:00200000:0.0:1787827241.394381:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008c0ef332 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394383:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001fec64a0 00000080:00200000:0.0:1787827241.394384:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:0.0:1787827241.394385:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394400:0:48385:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394403:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000010a73d79[3 00000000bdb38050 2 1 00000000746d8de4 0 0 18] 00000080:00200000:0.0:1787827241.394405:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006a231ae7 17ffffc0001001 3:0 ffff9a78859401d0 18 lru 00000080:00200000:0.0:1787827241.394408:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000055a9f643 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394410:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000010a73d79 00000080:00200000:0.0:1787827241.394411:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:0.0:1787827241.394412:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394421:0:48385:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394423:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000047e33f7d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 19] 00000080:00200000:0.0:1787827241.394425:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d24b6ca2 17ffffc0001001 3:0 ffff9a7885940740 19 lru 00000080:00200000:0.0:1787827241.394429:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000009ba0e89 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394431:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000047e33f7d 00000080:00200000:0.0:1787827241.394432:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:0.0:1787827241.394433:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394440:0:48385:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394443:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009074c586[3 00000000bdb38050 2 1 00000000746d8de4 0 0 20] 00000080:00200000:0.0:1787827241.394445:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000048e3c3c5 17ffffc0001001 3:0 ffff9a7885940e80 20 lru 00000080:00200000:0.0:1787827241.394448:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bfdd278b 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394449:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009074c586 00000080:00200000:0.0:1787827241.394450:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:0.0:1787827241.394451:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394461:0:48385:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394463:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000074ed090d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 21] 00000080:00200000:0.0:1787827241.394465:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a6ea964b 17ffffc0001001 3:0 ffff9a7885940488 21 lru 00000080:00200000:0.0:1787827241.394468:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006b6920c1 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394470:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000074ed090d 00000080:00200000:0.0:1787827241.394471:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:0.0:1787827241.394472:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394481:0:48385:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394484:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009a578d74[3 00000000bdb38050 2 1 00000000746d8de4 0 0 22] 00000080:00200000:0.0:1787827241.394486:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000016b2855e 17ffffc0001001 3:0 ffff9a7885940bc8 22 lru 00000080:00200000:0.0:1787827241.394490:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000066b10961 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394491:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009a578d74 00000080:00200000:0.0:1787827241.394492:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:0.0:1787827241.394493:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394514:0:48385:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394519:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002c3e420e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 23] 00000080:00200000:0.0:1787827241.394521:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ec87ffa7 17ffffc0001001 3:0 ffff9a7885940570 23 lru 00000080:00200000:0.0:1787827241.394524:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000afb3a21d 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394526:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002c3e420e 00000080:00200000:0.0:1787827241.394527:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:0.0:1787827241.394528:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394541:0:48385:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394543:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f60575f9[3 00000000bdb38050 2 1 00000000746d8de4 0 0 24] 00000080:00200000:0.0:1787827241.394545:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d85943ad 17ffffc0001001 3:0 ffff9a7885940828 24 lru 00000080:00200000:0.0:1787827241.394549:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000021709217 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394550:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f60575f9 00000080:00200000:0.0:1787827241.394551:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:0.0:1787827241.394553:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394564:0:48385:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394566:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000076c88cf8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 25] 00000080:00200000:0.0:1787827241.394568:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000048016c7a 17ffffc0001021 2:0 ffff9a7885940000 25 lru 00000080:00200000:0.0:1787827241.394572:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e438e86e 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394574:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000076c88cf8 00000080:00200000:0.0:1787827241.394574:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:0.0:1787827241.394576:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394584:0:48385:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394586:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000038e94150[3 00000000bdb38050 2 1 00000000746d8de4 0 0 26] 00000080:00200000:0.0:1787827241.394588:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000063a31cce 17ffffc0001001 3:0 ffff9a7885940cb0 26 lru 00000080:00200000:0.0:1787827241.394593:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ff0aa92e 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394594:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000038e94150 00000080:00200000:0.0:1787827241.394595:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:0.0:1787827241.394596:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394604:0:48385:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394607:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000eb6d2911[3 00000000bdb38050 2 1 00000000746d8de4 0 0 27] 00000080:00200000:0.0:1787827241.394609:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005a769d8e 17ffffc0001001 3:0 ffff9a7885940ae0 27 lru 00000080:00200000:0.0:1787827241.394612:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000895a83c3 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394614:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000eb6d2911 00000080:00200000:0.0:1787827241.394615:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:0.0:1787827241.394616:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394626:0:48385:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394628:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a1c7bf1b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 28] 00000080:00200000:0.0:1787827241.394630:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bbc9ca78 17ffffc0001001 3:0 ffff9a7885940658 28 lru 00000080:00200000:0.0:1787827241.394634:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000035381ed2 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394635:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a1c7bf1b 00000080:00200000:0.0:1787827241.394636:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:0.0:1787827241.394637:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394646:0:48385:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394648:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001161596d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 29] 00000080:00200000:0.0:1787827241.394650:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002b2138db 17ffffc0001001 3:0 ffff9a78859409f8 29 lru 00000080:00200000:0.0:1787827241.394654:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006db780ee 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394656:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001161596d 00000080:00200000:0.0:1787827241.394656:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:0.0:1787827241.394658:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394666:0:48385:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394668:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ac4fcf3f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 30] 00000080:00200000:0.0:1787827241.394670:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c66adc05 17ffffc0001001 3:0 ffff9a7885940910 30 lru 00000080:00200000:0.0:1787827241.394673:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000eb6fcb23 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394674:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ac4fcf3f 00000080:00200000:0.0:1787827241.394676:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:0.0:1787827241.394677:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394685:0:48385:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394688:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000041d8885a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 31] 00000080:00200000:0.0:1787827241.394690:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000094ef049b 17ffffc0001001 3:0 ffff9a7885940d98 31 lru 00000080:00200000:0.0:1787827241.394694:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002e1cc1c5 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394695:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000041d8885a 00000080:00200000:0.0:1787827241.394696:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:0.0:1787827241.394697:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394705:0:48385:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394707:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b34b1e87[3 00000000bdb38050 2 1 00000000746d8de4 0 0 32] 00000080:00200000:0.0:1787827241.394709:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006355bd98 17ffffc0001001 3:0 ffff9a78859403a0 32 lru 00000080:00200000:0.0:1787827241.394712:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007402a22a 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394713:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b34b1e87 00000080:00200000:0.0:1787827241.394714:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:0.0:1787827241.394715:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394723:0:48385:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394725:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006eb2919b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 33] 00000080:00200000:0.0:1787827241.394727:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008755c89e 17ffffc0001001 3:0 ffff9a78859400e8 33 lru 00000080:00200000:0.0:1787827241.394730:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000024f894be 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394731:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006eb2919b 00000080:00200000:0.0:1787827241.394732:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:0.0:1787827241.394733:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394741:0:48385:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394743:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000910ddc3b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 34] 00000080:00200000:0.0:1787827241.394745:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000029211178 17ffffc0001001 3:0 ffff9a78859402b8 34 lru 00000080:00200000:0.0:1787827241.394748:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003116ee11 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394749:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000910ddc3b 00000080:00200000:0.0:1787827241.394750:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:0.0:1787827241.394751:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394767:0:48385:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394770:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000dc03ffe[3 00000000bdb38050 2 1 00000000746d8de4 0 0 35] 00000080:00200000:0.0:1787827241.394772:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000d786ca7 17ffffc0001001 3:0 ffff9a7890940cb0 35 lru 00000080:00200000:0.0:1787827241.394776:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d0dfbd72 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394778:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000dc03ffe 00000080:00200000:0.0:1787827241.394778:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:0.0:1787827241.394780:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394788:0:48385:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394790:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009a822a7f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 36] 00000080:00200000:0.0:1787827241.394792:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a7ded5fe 17ffffc0001001 3:0 ffff9a7890940ae0 36 lru 00000080:00200000:0.0:1787827241.394795:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001a312353 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394797:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009a822a7f 00000080:00200000:0.0:1787827241.394798:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:0.0:1787827241.394799:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394807:0:48385:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394810:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000208bde92[3 00000000bdb38050 2 1 00000000746d8de4 0 0 37] 00000080:00200000:0.0:1787827241.394812:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000076d515c 17ffffc0001001 3:0 ffff9a7890940658 37 lru 00000080:00200000:0.0:1787827241.394815:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000917c598a 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394817:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000208bde92 00000080:00200000:0.0:1787827241.394817:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:0.0:1787827241.394818:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394826:0:48385:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394845:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000323a8b5e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 38] 00000080:00200000:0.0:1787827241.394849:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fd915f3d 17ffffc0001001 3:0 ffff9a78909409f8 38 lru 00000080:00200000:0.0:1787827241.394855:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000465d63e2 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394858:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000323a8b5e 00000080:00200000:0.0:1787827241.394859:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:0.0:1787827241.394861:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394879:0:48385:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394884:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006d150801[3 00000000bdb38050 2 1 00000000746d8de4 0 0 39] 00000080:00200000:0.0:1787827241.394887:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e7e71416 17ffffc0001001 3:0 ffff9a7890940910 39 lru 00000080:00200000:0.0:1787827241.394894:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d0e943e3 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394896:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006d150801 00000080:00200000:0.0:1787827241.394898:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:0.0:1787827241.394900:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394917:0:48385:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394922:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fb93ace1[3 00000000bdb38050 2 1 00000000746d8de4 0 0 40] 00000080:00200000:0.0:1787827241.394926:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000064f073d0 17ffffc0001021 2:0 ffff9a7890940d98 40 lru 00000080:00200000:0.0:1787827241.394932:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000020cf7819 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394934:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fb93ace1 00000080:00200000:0.0:1787827241.394936:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:0.0:1787827241.394937:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394948:0:48385:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394953:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000056d27c78[3 00000000bdb38050 2 1 00000000746d8de4 0 0 41] 00000080:00200000:0.0:1787827241.394956:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004ce2ee5a 17ffffc0001001 3:0 ffff9a78909403a0 41 lru 00000080:00200000:0.0:1787827241.394962:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fc870af0 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394965:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000056d27c78 00000080:00200000:0.0:1787827241.394966:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:0.0:1787827241.394968:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.394983:0:48385:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:0.0:1787827241.394986:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e4a00056[3 00000000bdb38050 2 1 00000000746d8de4 0 0 42] 00000080:00200000:0.0:1787827241.394989:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c8187447 17ffffc0001001 3:0 ffff9a78909400e8 42 lru 00000080:00200000:0.0:1787827241.394992:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000055f358bc 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.394994:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e4a00056 00000080:00200000:0.0:1787827241.394995:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:0.0:1787827241.394996:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395004:0:48385:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395007:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d759dd88[3 00000000bdb38050 2 1 00000000746d8de4 0 0 43] 00000080:00200000:0.0:1787827241.395009:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000047666192 17ffffc0001001 3:0 ffff9a78909402b8 43 lru 00000080:00200000:0.0:1787827241.395012:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b920b781 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395014:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d759dd88 00000080:00200000:0.0:1787827241.395015:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:0.0:1787827241.395016:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395024:0:48385:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395026:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000de6dfde4[3 00000000bdb38050 2 1 00000000746d8de4 0 0 44] 00000080:00200000:0.0:1787827241.395029:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bf035b41 17ffffc0001001 3:0 ffff9a78909401d0 44 lru 00000080:00200000:0.0:1787827241.395033:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002628ce81 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395034:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000de6dfde4 00000080:00200000:0.0:1787827241.395035:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:0.0:1787827241.395036:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395044:0:48385:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395047:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ba5d4c73[3 00000000bdb38050 2 1 00000000746d8de4 0 0 45] 00000080:00200000:0.0:1787827241.395048:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a3e2d1f4 17ffffc0001001 3:0 ffff9a7890940740 45 lru 00000080:00200000:0.0:1787827241.395052:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c3c6f2fa 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395054:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ba5d4c73 00000080:00200000:0.0:1787827241.395054:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:0.0:1787827241.395056:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395064:0:48385:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395067:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004eefbb3b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 46] 00000080:00200000:0.0:1787827241.395069:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b6cf46a5 17ffffc0001001 3:0 ffff9a7890940e80 46 lru 00000080:00200000:0.0:1787827241.395072:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000eebdbe8a 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395073:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004eefbb3b 00000080:00200000:0.0:1787827241.395074:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:0.0:1787827241.395075:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395083:0:48385:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395087:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c54e2bb7[3 00000000bdb38050 2 1 00000000746d8de4 0 0 47] 00000080:00200000:0.0:1787827241.395090:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002fa6fe93 17ffffc0001001 3:0 ffff9a7890940488 47 lru 00000080:00200000:0.0:1787827241.395095:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bac0303c 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395097:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c54e2bb7 00000080:00200000:0.0:1787827241.395098:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:0.0:1787827241.395100:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395114:0:48385:0:(rw26.c:875:ll_write_end()) pos 196608, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395119:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003d2e13d5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 48] 00000080:00200000:0.0:1787827241.395123:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000147393e0 17ffffc0001001 3:0 ffff9a7890940bc8 48 lru 00000080:00200000:0.0:1787827241.395129:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d0d43d54 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395132:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003d2e13d5 00000080:00200000:0.0:1787827241.395133:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:0.0:1787827241.395135:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 49 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395151:0:48385:0:(rw26.c:875:ll_write_end()) pos 200704, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395154:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ef3d1047[3 00000000bdb38050 2 1 00000000746d8de4 0 0 49] 00000080:00200000:0.0:1787827241.395156:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003c98ac77 17ffffc0001001 3:0 ffff9a7890940570 49 lru 00000080:00200000:0.0:1787827241.395160:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000090615635 49: 1< 0 - - > 2< 200704 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395161:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ef3d1047 00000080:00200000:0.0:1787827241.395162:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 50. 00000080:00200000:0.0:1787827241.395164:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 50 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395172:0:48385:0:(rw26.c:875:ll_write_end()) pos 204800, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395175:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a16afafd[3 00000000bdb38050 2 1 00000000746d8de4 0 0 50] 00000080:00200000:0.0:1787827241.395177:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004d8f81de 17ffffc0001001 3:0 ffff9a7890940828 50 lru 00000080:00200000:0.0:1787827241.395181:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e0c9a62b 50: 1< 0 - - > 2< 204800 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395182:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a16afafd 00000080:00200000:0.0:1787827241.395183:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 51. 00000080:00200000:0.0:1787827241.395184:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 51 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395192:0:48385:0:(rw26.c:875:ll_write_end()) pos 208896, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395195:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b7316c5d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 51] 00000080:00200000:0.0:1787827241.395196:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000015b07c3f 17ffffc0001001 3:0 ffff9a7890940000 51 lru 00000080:00200000:0.0:1787827241.395200:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005f77fa70 51: 1< 0 - - > 2< 208896 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395201:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b7316c5d 00000080:00200000:0.0:1787827241.395202:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 52. 00000080:00200000:0.0:1787827241.395203:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 52 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395218:0:48385:0:(rw26.c:875:ll_write_end()) pos 212992, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395220:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dbe9ea83[3 00000000bdb38050 2 1 00000000746d8de4 0 0 52] 00000080:00200000:0.0:1787827241.395222:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000094b175de 17ffffc0001001 3:0 ffff9a788875d828 52 lru 00000080:00200000:0.0:1787827241.395225:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000028560ad1 52: 1< 0 - - > 2< 212992 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395227:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dbe9ea83 00000080:00200000:0.0:1787827241.395228:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 53. 00000080:00200000:0.0:1787827241.395229:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 53 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395237:0:48385:0:(rw26.c:875:ll_write_end()) pos 217088, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395240:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000001bda938[3 00000000bdb38050 2 1 00000000746d8de4 0 0 53] 00000080:00200000:0.0:1787827241.395242:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f20ef1e0 17ffffc0001001 3:0 ffff9a788875d000 53 lru 00000080:00200000:0.0:1787827241.395246:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000032156d4c 53: 1< 0 - - > 2< 217088 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395248:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000001bda938 00000080:00200000:0.0:1787827241.395249:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 54. 00000080:00200000:0.0:1787827241.395250:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 54 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395259:0:48385:0:(rw26.c:875:ll_write_end()) pos 221184, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395261:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004738b103[3 00000000bdb38050 2 1 00000000746d8de4 0 0 54] 00000080:00200000:0.0:1787827241.395263:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000534e81e1 17ffffc0001001 3:0 ffff9a788875dcb0 54 lru 00000080:00200000:0.0:1787827241.395266:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000055d214d8 54: 1< 0 - - > 2< 221184 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395268:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004738b103 00000080:00200000:0.0:1787827241.395269:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 55. 00000080:00200000:0.0:1787827241.395270:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 55 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395281:0:48385:0:(rw26.c:875:ll_write_end()) pos 225280, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395284:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000800f614a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 55] 00000080:00200000:0.0:1787827241.395286:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000744edf5c 17ffffc0001021 2:0 ffff9a788875dae0 55 lru 00000080:00200000:0.0:1787827241.395290:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004f792f66 55: 1< 0 - - > 2< 225280 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395292:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000800f614a 00000080:00200000:0.0:1787827241.395292:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 56. 00000080:00200000:0.0:1787827241.395293:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 56 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395302:0:48385:0:(rw26.c:875:ll_write_end()) pos 229376, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395305:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006adc352d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 56] 00000080:00200000:0.0:1787827241.395307:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000060d3ac59 17ffffc0001001 3:0 ffff9a788875d658 56 lru 00000080:00200000:0.0:1787827241.395310:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000078727a02 56: 1< 0 - - > 2< 229376 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395311:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006adc352d 00000080:00200000:0.0:1787827241.395312:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 57. 00000080:00200000:0.0:1787827241.395313:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 57 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395322:0:48385:0:(rw26.c:875:ll_write_end()) pos 233472, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395324:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f8dd6b4e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 57] 00000080:00200000:0.0:1787827241.395326:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000040ad6a53 17ffffc0001001 3:0 ffff9a788875d9f8 57 lru 00000080:00200000:0.0:1787827241.395330:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000024526c8 57: 1< 0 - - > 2< 233472 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395331:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f8dd6b4e 00000080:00200000:0.0:1787827241.395332:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 58. 00000080:00200000:0.0:1787827241.395334:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 58 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395342:0:48385:0:(rw26.c:875:ll_write_end()) pos 237568, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395345:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000042f8114b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 58] 00000080:00200000:0.0:1787827241.395347:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000009b4c679 17ffffc0001001 3:0 ffff9a788875d910 58 lru 00000080:00200000:0.0:1787827241.395350:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000006100bfd 58: 1< 0 - - > 2< 237568 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395352:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000042f8114b 00000080:00200000:0.0:1787827241.395353:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 59. 00000080:00200000:0.0:1787827241.395354:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 59 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395362:0:48385:0:(rw26.c:875:ll_write_end()) pos 241664, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395365:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b6687d8a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 59] 00000080:00200000:0.0:1787827241.395366:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000299ec1e4 17ffffc0001001 3:0 ffff9a788875dd98 59 lru 00000080:00200000:0.0:1787827241.395370:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000392fd6d8 59: 1< 0 - - > 2< 241664 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395371:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b6687d8a 00000080:00200000:0.0:1787827241.395371:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 60. 00000080:00200000:0.0:1787827241.395372:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 60 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395380:0:48385:0:(rw26.c:875:ll_write_end()) pos 245760, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395383:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000677b4617[3 00000000bdb38050 2 1 00000000746d8de4 0 0 60] 00000080:00200000:0.0:1787827241.395385:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000079d8925c 17ffffc0001001 3:0 ffff9a788875d3a0 60 lru 00000080:00200000:0.0:1787827241.395388:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000067762508 60: 1< 0 - - > 2< 245760 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395390:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000677b4617 00000080:00200000:0.0:1787827241.395391:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 61. 00000080:00200000:0.0:1787827241.395392:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 61 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395400:0:48385:0:(rw26.c:875:ll_write_end()) pos 249856, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395402:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005e83048d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 61] 00000080:00200000:0.0:1787827241.395404:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000af8bb444 17ffffc0001001 3:0 ffff9a788875d0e8 61 lru 00000080:00200000:0.0:1787827241.395407:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000235f4e8c 61: 1< 0 - - > 2< 249856 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395409:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005e83048d 00000080:00200000:0.0:1787827241.395410:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 62. 00000080:00200000:0.0:1787827241.395411:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 62 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395422:0:48385:0:(rw26.c:875:ll_write_end()) pos 253952, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395424:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009ef8e6a6[3 00000000bdb38050 2 1 00000000746d8de4 0 0 62] 00000080:00200000:0.0:1787827241.395427:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c8fa2aa3 17ffffc0001001 3:0 ffff9a788875d2b8 62 lru 00000080:00200000:0.0:1787827241.395430:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bb863899 62: 1< 0 - - > 2< 253952 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395432:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009ef8e6a6 00000080:00200000:0.0:1787827241.395433:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 63. 00000080:00200000:0.0:1787827241.395434:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 63 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395443:0:48385:0:(rw26.c:875:ll_write_end()) pos 258048, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395446:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001b1cd211[3 00000000bdb38050 2 1 00000000746d8de4 0 0 63] 00000080:00200000:0.0:1787827241.395448:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000955cf074 17ffffc0001001 3:0 ffff9a788875d1d0 63 lru 00000080:00200000:0.0:1787827241.395451:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c5ac11f8 63: 1< 0 - - > 2< 258048 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395453:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001b1cd211 00000080:00200000:0.0:1787827241.395454:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 64. 00000080:00200000:0.0:1787827241.395455:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 64 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395467:0:48385:0:(rw26.c:875:ll_write_end()) pos 262144, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395470:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000640277ad[3 00000000bdb38050 2 1 00000000746d8de4 0 0 64] 00000080:00200000:0.0:1787827241.395472:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b421faca 17ffffc0001001 3:0 ffff9a788875d740 64 lru 00000080:00200000:0.0:1787827241.395476:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e7f65af2 64: 1< 0 - - > 2< 262144 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395477:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000640277ad 00000080:00200000:0.0:1787827241.395478:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 65. 00000080:00200000:0.0:1787827241.395479:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 65 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395488:0:48385:0:(rw26.c:875:ll_write_end()) pos 266240, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395490:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bee80ecb[3 00000000bdb38050 2 1 00000000746d8de4 0 0 65] 00000080:00200000:0.0:1787827241.395492:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006e592954 17ffffc0001001 3:0 ffff9a788875de80 65 lru 00000080:00200000:0.0:1787827241.395495:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000059625b59 65: 1< 0 - - > 2< 266240 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395497:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bee80ecb 00000080:00200000:0.0:1787827241.395498:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 66. 00000080:00200000:0.0:1787827241.395499:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 66 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395527:0:48385:0:(rw26.c:875:ll_write_end()) pos 270336, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395530:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c7800bc0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 66] 00000080:00200000:0.0:1787827241.395532:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c662d369 17ffffc0001001 3:0 ffff9a788875d488 66 lru 00000080:00200000:0.0:1787827241.395536:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bae6a7dd 66: 1< 0 - - > 2< 270336 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395538:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c7800bc0 00000080:00200000:0.0:1787827241.395539:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 67. 00000080:00200000:0.0:1787827241.395540:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 67 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395549:0:48385:0:(rw26.c:875:ll_write_end()) pos 274432, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395552:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008981a0ad[3 00000000bdb38050 2 1 00000000746d8de4 0 0 67] 00000080:00200000:0.0:1787827241.395554:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bfa838a2 17ffffc0001001 3:0 ffff9a788875dbc8 67 lru 00000080:00200000:0.0:1787827241.395558:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000798911ac 67: 1< 0 - - > 2< 274432 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395559:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008981a0ad 00000080:00200000:0.0:1787827241.395560:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 68. 00000080:00200000:0.0:1787827241.395561:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 68 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395569:0:48385:0:(rw26.c:875:ll_write_end()) pos 278528, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395572:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009c8b6771[3 00000000bdb38050 2 1 00000000746d8de4 0 0 68] 00000080:00200000:0.0:1787827241.395573:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002892c513 17ffffc0001001 3:0 ffff9a788875d570 68 lru 00000080:00200000:0.0:1787827241.395577:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000080e45019 68: 1< 0 - - > 2< 278528 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395578:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009c8b6771 00000080:00200000:0.0:1787827241.395579:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 69. 00000080:00200000:0.0:1787827241.395580:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 69 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395594:0:48385:0:(rw26.c:875:ll_write_end()) pos 282624, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395597:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000efa5c6f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 69] 00000080:00200000:0.0:1787827241.395598:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cd80db9f 17ffffc0001001 3:0 ffff9a78887cf740 69 lru 00000080:00200000:0.0:1787827241.395602:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d4f30d1e 69: 1< 0 - - > 2< 282624 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395603:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000efa5c6f 00000080:00200000:0.0:1787827241.395604:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 70. 00000080:00200000:0.0:1787827241.395605:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 70 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395618:0:48385:0:(rw26.c:875:ll_write_end()) pos 286720, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395620:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002f185616[3 00000000bdb38050 2 1 00000000746d8de4 0 0 70] 00000080:00200000:0.0:1787827241.395622:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000032259c62 17ffffc0001021 2:0 ffff9a78887cfe80 70 lru 00000080:00200000:0.0:1787827241.395625:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000018f164d7 70: 1< 0 - - > 2< 286720 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395627:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002f185616 00000080:00200000:0.0:1787827241.395628:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 71. 00000080:00200000:0.0:1787827241.395629:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 71 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395639:0:48385:0:(rw26.c:875:ll_write_end()) pos 290816, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395641:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dbc8dab3[3 00000000bdb38050 2 1 00000000746d8de4 0 0 71] 00000080:00200000:0.0:1787827241.395644:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003d50edd4 17ffffc0001001 3:0 ffff9a78887cf488 71 lru 00000080:00200000:0.0:1787827241.395647:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000020de4d44 71: 1< 0 - - > 2< 290816 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395649:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dbc8dab3 00000080:00200000:0.0:1787827241.395650:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 72. 00000080:00200000:0.0:1787827241.395651:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 72 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395659:0:48385:0:(rw26.c:875:ll_write_end()) pos 294912, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395662:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a704a645[3 00000000bdb38050 2 1 00000000746d8de4 0 0 72] 00000080:00200000:0.0:1787827241.395664:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bceff124 17ffffc0001001 3:0 ffff9a78887cfbc8 72 lru 00000080:00200000:0.0:1787827241.395667:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b297b80d 72: 1< 0 - - > 2< 294912 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395669:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a704a645 00000080:00200000:0.0:1787827241.395670:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 73. 00000080:00200000:0.0:1787827241.395671:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 73 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395679:0:48385:0:(rw26.c:875:ll_write_end()) pos 299008, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395682:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000821b222[3 00000000bdb38050 2 1 00000000746d8de4 0 0 73] 00000080:00200000:0.0:1787827241.395684:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007794c17a 17ffffc0001001 3:0 ffff9a78887cf570 73 lru 00000080:00200000:0.0:1787827241.395687:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000226b44a8 73: 1< 0 - - > 2< 299008 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395689:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000821b222 00000080:00200000:0.0:1787827241.395690:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 74. 00000080:00200000:0.0:1787827241.395691:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 74 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395700:0:48385:0:(rw26.c:875:ll_write_end()) pos 303104, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395702:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b7999e59[3 00000000bdb38050 2 1 00000000746d8de4 0 0 74] 00000080:00200000:0.0:1787827241.395704:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000046343933 17ffffc0001001 3:0 ffff9a78887cf828 74 lru 00000080:00200000:0.0:1787827241.395708:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004fe346bf 74: 1< 0 - - > 2< 303104 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395709:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b7999e59 00000080:00200000:0.0:1787827241.395709:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 75. 00000080:00200000:0.0:1787827241.395711:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 75 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395719:0:48385:0:(rw26.c:875:ll_write_end()) pos 307200, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395721:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000092fa9804[3 00000000bdb38050 2 1 00000000746d8de4 0 0 75] 00000080:00200000:0.0:1787827241.395723:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f94aac01 17ffffc0001001 3:0 ffff9a78887cf000 75 lru 00000080:00200000:0.0:1787827241.395727:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002b177709 75: 1< 0 - - > 2< 307200 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395728:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000092fa9804 00000080:00200000:0.0:1787827241.395729:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 76. 00000080:00200000:0.0:1787827241.395730:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 76 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395738:0:48385:0:(rw26.c:875:ll_write_end()) pos 311296, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395741:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cf96e32d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 76] 00000080:00200000:0.0:1787827241.395743:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004bc2b43f 17ffffc0001001 3:0 ffff9a78887cfcb0 76 lru 00000080:00200000:0.0:1787827241.395746:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006085918e 76: 1< 0 - - > 2< 311296 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395748:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cf96e32d 00000080:00200000:0.0:1787827241.395749:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 77. 00000080:00200000:0.0:1787827241.395750:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 77 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395760:0:48385:0:(rw26.c:875:ll_write_end()) pos 315392, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395762:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d7c6c210[3 00000000bdb38050 2 1 00000000746d8de4 0 0 77] 00000080:00200000:0.0:1787827241.395764:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d67a286a 17ffffc0001001 3:0 ffff9a78887cfae0 77 lru 00000080:00200000:0.0:1787827241.395768:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000669676e9 77: 1< 0 - - > 2< 315392 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395770:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d7c6c210 00000080:00200000:0.0:1787827241.395770:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 78. 00000080:00200000:0.0:1787827241.395772:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 78 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395780:0:48385:0:(rw26.c:875:ll_write_end()) pos 319488, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395783:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007a380ce0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 78] 00000080:00200000:0.0:1787827241.395784:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b898adc3 17ffffc0001001 3:0 ffff9a78887cf658 78 lru 00000080:00200000:0.0:1787827241.395788:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cd185820 78: 1< 0 - - > 2< 319488 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395789:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007a380ce0 00000080:00200000:0.0:1787827241.395790:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 79. 00000080:00200000:0.0:1787827241.395791:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 79 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395799:0:48385:0:(rw26.c:875:ll_write_end()) pos 323584, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395801:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001836fe1b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 79] 00000080:00200000:0.0:1787827241.395803:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004ee1deca 17ffffc0001001 3:0 ffff9a78887cf9f8 79 lru 00000080:00200000:0.0:1787827241.395806:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005f1fb717 79: 1< 0 - - > 2< 323584 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395807:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001836fe1b 00000080:00200000:0.0:1787827241.395808:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 80. 00000080:00200000:0.0:1787827241.395809:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 80 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395817:0:48385:0:(rw26.c:875:ll_write_end()) pos 327680, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395820:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000987e4791[3 00000000bdb38050 2 1 00000000746d8de4 0 0 80] 00000080:00200000:0.0:1787827241.395822:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cbf16738 17ffffc0001001 3:0 ffff9a78887cf910 80 lru 00000080:00200000:0.0:1787827241.395826:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c2574dce 80: 1< 0 - - > 2< 327680 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395848:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000987e4791 00000080:00200000:0.0:1787827241.395850:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 81. 00000080:00200000:0.0:1787827241.395852:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 81 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395871:0:48385:0:(rw26.c:875:ll_write_end()) pos 331776, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395876:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000674bb839[3 00000000bdb38050 2 1 00000000746d8de4 0 0 81] 00000080:00200000:0.0:1787827241.395880:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000601e42bf 17ffffc0001001 3:0 ffff9a78887cfd98 81 lru 00000080:00200000:0.0:1787827241.395885:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000417baf7a 81: 1< 0 - - > 2< 331776 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395888:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000674bb839 00000080:00200000:0.0:1787827241.395889:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 82. 00000080:00200000:0.0:1787827241.395891:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 82 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395907:0:48385:0:(rw26.c:875:ll_write_end()) pos 335872, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395912:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e0cfc8dd[3 00000000bdb38050 2 1 00000000746d8de4 0 0 82] 00000080:00200000:0.0:1787827241.395916:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f3ec0525 17ffffc0001001 3:0 ffff9a78887cf3a0 82 lru 00000080:00200000:0.0:1787827241.395922:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000beb13b4f 82: 1< 0 - - > 2< 335872 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395925:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e0cfc8dd 00000080:00200000:0.0:1787827241.395926:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 83. 00000080:00200000:0.0:1787827241.395928:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 83 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395940:0:48385:0:(rw26.c:875:ll_write_end()) pos 339968, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395944:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000f7de32a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 83] 00000080:00200000:0.0:1787827241.395948:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000089a1c5fc 17ffffc0001001 3:0 ffff9a78887cf0e8 83 lru 00000080:00200000:0.0:1787827241.395953:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e2fe39f7 83: 1< 0 - - > 2< 339968 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395955:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000f7de32a 00000080:00200000:0.0:1787827241.395956:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 84. 00000080:00200000:0.0:1787827241.395957:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 84 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395969:0:48385:0:(rw26.c:875:ll_write_end()) pos 344064, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395972:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000132e1138[3 00000000bdb38050 2 1 00000000746d8de4 0 0 84] 00000080:00200000:0.0:1787827241.395974:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005a5efee5 17ffffc0001001 3:0 ffff9a78887cf2b8 84 lru 00000080:00200000:0.0:1787827241.395978:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e698cacf 84: 1< 0 - - > 2< 344064 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.395979:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000132e1138 00000080:00200000:0.0:1787827241.395981:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 85. 00000080:00200000:0.0:1787827241.395982:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 85 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.395993:0:48385:0:(rw26.c:875:ll_write_end()) pos 348160, len 4096, copied 4096 00000080:00200000:0.0:1787827241.395996:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004df002df[3 00000000bdb38050 2 1 00000000746d8de4 0 0 85] 00000080:00200000:0.0:1787827241.395998:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b981d4c0 17ffffc0001021 2:0 ffff9a78887cf1d0 85 lru 00000080:00200000:0.0:1787827241.396001:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000051ec967b 85: 1< 0 - - > 2< 348160 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396003:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004df002df 00000080:00200000:0.0:1787827241.396004:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 86. 00000080:00200000:0.0:1787827241.396005:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 86 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396020:0:48385:0:(rw26.c:875:ll_write_end()) pos 352256, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396023:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fa3bd17e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 86] 00000080:00200000:0.0:1787827241.396025:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e57a5018 17ffffc0001001 3:0 ffff9a788801dd98 86 lru 00000080:00200000:0.0:1787827241.396028:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000088e34f9c 86: 1< 0 - - > 2< 352256 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396030:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fa3bd17e 00000080:00200000:0.0:1787827241.396030:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 87. 00000080:00200000:0.0:1787827241.396032:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 87 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396040:0:48385:0:(rw26.c:875:ll_write_end()) pos 356352, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396043:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006f6823ed[3 00000000bdb38050 2 1 00000000746d8de4 0 0 87] 00000080:00200000:0.0:1787827241.396045:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000eaf0b658 17ffffc0001001 3:0 ffff9a788801d3a0 87 lru 00000080:00200000:0.0:1787827241.396049:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f305e033 87: 1< 0 - - > 2< 356352 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396050:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006f6823ed 00000080:00200000:0.0:1787827241.396051:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 88. 00000080:00200000:0.0:1787827241.396052:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 88 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396061:0:48385:0:(rw26.c:875:ll_write_end()) pos 360448, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396063:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a31dd349[3 00000000bdb38050 2 1 00000000746d8de4 0 0 88] 00000080:00200000:0.0:1787827241.396065:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000045c51933 17ffffc0001001 3:0 ffff9a788801d0e8 88 lru 00000080:00200000:0.0:1787827241.396068:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000028eb19e9 88: 1< 0 - - > 2< 360448 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396070:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a31dd349 00000080:00200000:0.0:1787827241.396071:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 89. 00000080:00200000:0.0:1787827241.396072:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 89 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396081:0:48385:0:(rw26.c:875:ll_write_end()) pos 364544, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396085:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006b9c0f74[3 00000000bdb38050 2 1 00000000746d8de4 0 0 89] 00000080:00200000:0.0:1787827241.396089:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000241a7477 17ffffc0001001 3:0 ffff9a788801d2b8 89 lru 00000080:00200000:0.0:1787827241.396095:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004dccf948 89: 1< 0 - - > 2< 364544 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396097:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006b9c0f74 00000080:00200000:0.0:1787827241.396098:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 90. 00000080:00200000:0.0:1787827241.396100:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 90 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396116:0:48385:0:(rw26.c:875:ll_write_end()) pos 368640, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396120:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003f879fce[3 00000000bdb38050 2 1 00000000746d8de4 0 0 90] 00000080:00200000:0.0:1787827241.396124:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009f12302a 17ffffc0001001 3:0 ffff9a788801d1d0 90 lru 00000080:00200000:0.0:1787827241.396130:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001f88a1b8 90: 1< 0 - - > 2< 368640 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396133:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003f879fce 00000080:00200000:0.0:1787827241.396134:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 91. 00000080:00200000:0.0:1787827241.396136:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 91 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396149:0:48385:0:(rw26.c:875:ll_write_end()) pos 372736, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396152:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000060904d79[3 00000000bdb38050 2 1 00000000746d8de4 0 0 91] 00000080:00200000:0.0:1787827241.396154:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000027590358 17ffffc0001001 3:0 ffff9a788801d740 91 lru 00000080:00200000:0.0:1787827241.396158:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002da98388 91: 1< 0 - - > 2< 372736 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396160:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000060904d79 00000080:00200000:0.0:1787827241.396160:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 92. 00000080:00200000:0.0:1787827241.396161:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 92 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396170:0:48385:0:(rw26.c:875:ll_write_end()) pos 376832, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396172:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cc891e25[3 00000000bdb38050 2 1 00000000746d8de4 0 0 92] 00000080:00200000:0.0:1787827241.396174:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009fcd9b26 17ffffc0001001 3:0 ffff9a788801de80 92 lru 00000080:00200000:0.0:1787827241.396178:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000072ae4739 92: 1< 0 - - > 2< 376832 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396180:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cc891e25 00000080:00200000:0.0:1787827241.396181:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 93. 00000080:00200000:0.0:1787827241.396182:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 93 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396190:0:48385:0:(rw26.c:875:ll_write_end()) pos 380928, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396193:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dde0b523[3 00000000bdb38050 2 1 00000000746d8de4 0 0 93] 00000080:00200000:0.0:1787827241.396194:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000008c2a422 17ffffc0001001 3:0 ffff9a788801d488 93 lru 00000080:00200000:0.0:1787827241.396198:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b3509b2a 93: 1< 0 - - > 2< 380928 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396199:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dde0b523 00000080:00200000:0.0:1787827241.396200:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 94. 00000080:00200000:0.0:1787827241.396201:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 94 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396210:0:48385:0:(rw26.c:875:ll_write_end()) pos 385024, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396212:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000abd43771[3 00000000bdb38050 2 1 00000000746d8de4 0 0 94] 00000080:00200000:0.0:1787827241.396214:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000042dcbf9c 17ffffc0001001 3:0 ffff9a788801dbc8 94 lru 00000080:00200000:0.0:1787827241.396218:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000aaaad638 94: 1< 0 - - > 2< 385024 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396219:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000abd43771 00000080:00200000:0.0:1787827241.396220:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 95. 00000080:00200000:0.0:1787827241.396221:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 95 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396229:0:48385:0:(rw26.c:875:ll_write_end()) pos 389120, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396231:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002a67edf5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 95] 00000080:00200000:0.0:1787827241.396233:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000047fd7d37 17ffffc0001001 3:0 ffff9a788801d570 95 lru 00000080:00200000:0.0:1787827241.396236:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000426da945 95: 1< 0 - - > 2< 389120 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396238:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002a67edf5 00000080:00200000:0.0:1787827241.396238:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 96. 00000080:00200000:0.0:1787827241.396239:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 96 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396247:0:48385:0:(rw26.c:875:ll_write_end()) pos 393216, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396250:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e624b600[3 00000000bdb38050 2 1 00000000746d8de4 0 0 96] 00000080:00200000:0.0:1787827241.396252:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000de19e52 17ffffc0001001 3:0 ffff9a788801d828 96 lru 00000080:00200000:0.0:1787827241.396256:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003093b116 96: 1< 0 - - > 2< 393216 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396257:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e624b600 00000080:00200000:0.0:1787827241.396258:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 97. 00000080:00200000:0.0:1787827241.396259:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 97 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396267:0:48385:0:(rw26.c:875:ll_write_end()) pos 397312, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396269:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000621ba63e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 97] 00000080:00200000:0.0:1787827241.396271:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a936d785 17ffffc0001001 3:0 ffff9a788801d000 97 lru 00000080:00200000:0.0:1787827241.396275:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ee08bcef 97: 1< 0 - - > 2< 397312 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396276:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000621ba63e 00000080:00200000:0.0:1787827241.396277:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 98. 00000080:00200000:0.0:1787827241.396278:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 98 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396288:0:48385:0:(rw26.c:875:ll_write_end()) pos 401408, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396291:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b386c6e2[3 00000000bdb38050 2 1 00000000746d8de4 0 0 98] 00000080:00200000:0.0:1787827241.396293:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001ef83284 17ffffc0001001 3:0 ffff9a788801dcb0 98 lru 00000080:00200000:0.0:1787827241.396297:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000bedd31d 98: 1< 0 - - > 2< 401408 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396298:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b386c6e2 00000080:00200000:0.0:1787827241.396299:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 99. 00000080:00200000:0.0:1787827241.396301:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 99 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396309:0:48385:0:(rw26.c:875:ll_write_end()) pos 405504, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396311:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d9b3ece8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 99] 00000080:00200000:0.0:1787827241.396313:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005685f80b 17ffffc0001001 3:0 ffff9a788801dae0 99 lru 00000080:00200000:0.0:1787827241.396316:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a79b6386 99: 1< 0 - - > 2< 405504 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396317:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d9b3ece8 00000080:00200000:0.0:1787827241.396318:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 100. 00000080:00200000:0.0:1787827241.396320:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 100 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396330:0:48385:0:(rw26.c:875:ll_write_end()) pos 409600, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396333:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005a239473[3 00000000bdb38050 2 1 00000000746d8de4 0 0 100] 00000080:00200000:0.0:1787827241.396335:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cc8fb856 17ffffc0001021 2:0 ffff9a788801d658 100 lru 00000080:00200000:0.0:1787827241.396339:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ef83746d 100: 1< 0 - - > 2< 409600 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396341:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005a239473 00000080:00200000:0.0:1787827241.396342:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 101. 00000080:00200000:0.0:1787827241.396343:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 101 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396350:0:48385:0:(rw26.c:875:ll_write_end()) pos 413696, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396353:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e711a39b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 101] 00000080:00200000:0.0:1787827241.396354:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001041a8fe 17ffffc0001001 3:0 ffff9a788801d9f8 101 lru 00000080:00200000:0.0:1787827241.396358:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007bcd8eca 101: 1< 0 - - > 2< 413696 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396359:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e711a39b 00000080:00200000:0.0:1787827241.396360:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 102. 00000080:00200000:0.0:1787827241.396361:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 102 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396370:0:48385:0:(rw26.c:875:ll_write_end()) pos 417792, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396372:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b46da0f5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 102] 00000080:00200000:0.0:1787827241.396374:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005dd11266 17ffffc0001001 3:0 ffff9a788801d910 102 lru 00000080:00200000:0.0:1787827241.396378:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fd74d738 102: 1< 0 - - > 2< 417792 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396380:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b46da0f5 00000080:00200000:0.0:1787827241.396381:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 103. 00000080:00200000:0.0:1787827241.396383:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 103 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396397:0:48385:0:(rw26.c:875:ll_write_end()) pos 421888, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396400:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006e078d43[3 00000000bdb38050 2 1 00000000746d8de4 0 0 103] 00000080:00200000:0.0:1787827241.396402:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000567fba97 17ffffc0001001 3:0 ffff9a78860c1e80 103 lru 00000080:00200000:0.0:1787827241.396406:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000baaa104b 103: 1< 0 - - > 2< 421888 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396407:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006e078d43 00000080:00200000:0.0:1787827241.396408:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 104. 00000080:00200000:0.0:1787827241.396409:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 104 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396417:0:48385:0:(rw26.c:875:ll_write_end()) pos 425984, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396419:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000087f9ca92[3 00000000bdb38050 2 1 00000000746d8de4 0 0 104] 00000080:00200000:0.0:1787827241.396421:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008360dbf9 17ffffc0001001 3:0 ffff9a78860c1488 104 lru 00000080:00200000:0.0:1787827241.396424:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000daab6feb 104: 1< 0 - - > 2< 425984 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396426:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000087f9ca92 00000080:00200000:0.0:1787827241.396427:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 105. 00000080:00200000:0.0:1787827241.396428:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 105 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396438:0:48385:0:(rw26.c:875:ll_write_end()) pos 430080, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396440:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c824b205[3 00000000bdb38050 2 1 00000000746d8de4 0 0 105] 00000080:00200000:0.0:1787827241.396442:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000251fcc4e 17ffffc0001001 3:0 ffff9a78860c1bc8 105 lru 00000080:00200000:0.0:1787827241.396446:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008742eed6 105: 1< 0 - - > 2< 430080 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396448:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c824b205 00000080:00200000:0.0:1787827241.396449:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 106. 00000080:00200000:0.0:1787827241.396450:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 106 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396458:0:48385:0:(rw26.c:875:ll_write_end()) pos 434176, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396461:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006e429552[3 00000000bdb38050 2 1 00000000746d8de4 0 0 106] 00000080:00200000:0.0:1787827241.396463:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000215eeb5e 17ffffc0001001 3:0 ffff9a78860c1570 106 lru 00000080:00200000:0.0:1787827241.396466:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f759b3d7 106: 1< 0 - - > 2< 434176 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396467:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006e429552 00000080:00200000:0.0:1787827241.396468:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 107. 00000080:00200000:0.0:1787827241.396469:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 107 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396478:0:48385:0:(rw26.c:875:ll_write_end()) pos 438272, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396481:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bfeacdfe[3 00000000bdb38050 2 1 00000000746d8de4 0 0 107] 00000080:00200000:0.0:1787827241.396483:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000054ba982e 17ffffc0001001 3:0 ffff9a78860c1828 107 lru 00000080:00200000:0.0:1787827241.396487:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003ede8a57 107: 1< 0 - - > 2< 438272 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396488:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bfeacdfe 00000080:00200000:0.0:1787827241.396489:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 108. 00000080:00200000:0.0:1787827241.396490:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 108 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396498:0:48385:0:(rw26.c:875:ll_write_end()) pos 442368, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396500:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000457e25bf[3 00000000bdb38050 2 1 00000000746d8de4 0 0 108] 00000080:00200000:0.0:1787827241.396516:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002c891fe7 17ffffc0001001 3:0 ffff9a78860c1000 108 lru 00000080:00200000:0.0:1787827241.396520:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000069e29af0 108: 1< 0 - - > 2< 442368 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396522:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000457e25bf 00000080:00200000:0.0:1787827241.396523:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 109. 00000080:00200000:0.0:1787827241.396524:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 109 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396536:0:48385:0:(rw26.c:875:ll_write_end()) pos 446464, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396539:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bb1cc1a6[3 00000000bdb38050 2 1 00000000746d8de4 0 0 109] 00000080:00200000:0.0:1787827241.396541:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000030d5288b 17ffffc0001001 3:0 ffff9a78860c1cb0 109 lru 00000080:00200000:0.0:1787827241.396545:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ae556dc9 109: 1< 0 - - > 2< 446464 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396546:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bb1cc1a6 00000080:00200000:0.0:1787827241.396547:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 110. 00000080:00200000:0.0:1787827241.396548:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 110 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396557:0:48385:0:(rw26.c:875:ll_write_end()) pos 450560, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396560:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000209f16b7[3 00000000bdb38050 2 1 00000000746d8de4 0 0 110] 00000080:00200000:0.0:1787827241.396562:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000041358814 17ffffc0001001 3:0 ffff9a78860c1ae0 110 lru 00000080:00200000:0.0:1787827241.396565:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004c65ad33 110: 1< 0 - - > 2< 450560 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396567:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000209f16b7 00000080:00200000:0.0:1787827241.396568:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 111. 00000080:00200000:0.0:1787827241.396569:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 111 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396578:0:48385:0:(rw26.c:875:ll_write_end()) pos 454656, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396580:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000042bce2b4[3 00000000bdb38050 2 1 00000000746d8de4 0 0 111] 00000080:00200000:0.0:1787827241.396582:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000864004a9 17ffffc0001001 3:0 ffff9a78860c1658 111 lru 00000080:00200000:0.0:1787827241.396586:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fa2dd495 111: 1< 0 - - > 2< 454656 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396588:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000042bce2b4 00000080:00200000:0.0:1787827241.396589:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 112. 00000080:00200000:0.0:1787827241.396590:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 112 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396600:0:48385:0:(rw26.c:875:ll_write_end()) pos 458752, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396603:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000927595aa[3 00000000bdb38050 2 1 00000000746d8de4 0 0 112] 00000080:00200000:0.0:1787827241.396605:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e9ff1961 17ffffc0001001 3:0 ffff9a78860c19f8 112 lru 00000080:00200000:0.0:1787827241.396609:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e2a88d9e 112: 1< 0 - - > 2< 458752 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396610:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000927595aa 00000080:00200000:0.0:1787827241.396611:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 113. 00000080:00200000:0.0:1787827241.396612:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 113 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396620:0:48385:0:(rw26.c:875:ll_write_end()) pos 462848, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396623:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f2bb6bd9[3 00000000bdb38050 2 1 00000000746d8de4 0 0 113] 00000080:00200000:0.0:1787827241.396624:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f5a3b5dc 17ffffc0001001 3:0 ffff9a78860c1910 113 lru 00000080:00200000:0.0:1787827241.396628:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000be6d5be5 113: 1< 0 - - > 2< 462848 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396629:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f2bb6bd9 00000080:00200000:0.0:1787827241.396630:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 114. 00000080:00200000:0.0:1787827241.396631:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 114 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396640:0:48385:0:(rw26.c:875:ll_write_end()) pos 466944, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396643:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e57c363e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 114] 00000080:00200000:0.0:1787827241.396645:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d09f784c 17ffffc0001001 3:0 ffff9a78860c1d98 114 lru 00000080:00200000:0.0:1787827241.396648:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008a8eb25d 114: 1< 0 - - > 2< 466944 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396650:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e57c363e 00000080:00200000:0.0:1787827241.396651:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 115. 00000080:00200000:0.0:1787827241.396652:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 115 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396662:0:48385:0:(rw26.c:875:ll_write_end()) pos 471040, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396665:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008bfba20b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 115] 00000080:00200000:0.0:1787827241.396667:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000110f2e9f 17ffffc0001021 2:0 ffff9a78860c13a0 115 lru 00000080:00200000:0.0:1787827241.396670:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000079c47151 115: 1< 0 - - > 2< 471040 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396671:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008bfba20b 00000080:00200000:0.0:1787827241.396672:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 116. 00000080:00200000:0.0:1787827241.396673:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 116 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396682:0:48385:0:(rw26.c:875:ll_write_end()) pos 475136, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396685:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000efaa9ca9[3 00000000bdb38050 2 1 00000000746d8de4 0 0 116] 00000080:00200000:0.0:1787827241.396687:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a15f6f45 17ffffc0001001 3:0 ffff9a78860c10e8 116 lru 00000080:00200000:0.0:1787827241.396691:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005e9176f1 116: 1< 0 - - > 2< 475136 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396693:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000efaa9ca9 00000080:00200000:0.0:1787827241.396693:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 117. 00000080:00200000:0.0:1787827241.396694:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 117 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396702:0:48385:0:(rw26.c:875:ll_write_end()) pos 479232, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396705:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000af4aa28e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 117] 00000080:00200000:0.0:1787827241.396707:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008ca408f4 17ffffc0001001 3:0 ffff9a78860c12b8 117 lru 00000080:00200000:0.0:1787827241.396710:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e06c6bce 117: 1< 0 - - > 2< 479232 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396712:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000af4aa28e 00000080:00200000:0.0:1787827241.396712:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 118. 00000080:00200000:0.0:1787827241.396714:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 118 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396722:0:48385:0:(rw26.c:875:ll_write_end()) pos 483328, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396724:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001b219263[3 00000000bdb38050 2 1 00000000746d8de4 0 0 118] 00000080:00200000:0.0:1787827241.396726:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000056db37d5 17ffffc0001001 3:0 ffff9a78860c11d0 118 lru 00000080:00200000:0.0:1787827241.396729:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000368fd77a 118: 1< 0 - - > 2< 483328 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396730:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001b219263 00000080:00200000:0.0:1787827241.396731:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 119. 00000080:00200000:0.0:1787827241.396732:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 119 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396741:0:48385:0:(rw26.c:875:ll_write_end()) pos 487424, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396744:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d24f772e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 119] 00000080:00200000:0.0:1787827241.396745:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009718eba1 17ffffc0001001 3:0 ffff9a78860c1740 119 lru 00000080:00200000:0.0:1787827241.396749:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004c16556f 119: 1< 0 - - > 2< 487424 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396750:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d24f772e 00000080:00200000:0.0:1787827241.396751:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 120. 00000080:00200000:0.0:1787827241.396752:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 120 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396766:0:48385:0:(rw26.c:875:ll_write_end()) pos 491520, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396769:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007e44569c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 120] 00000080:00200000:0.0:1787827241.396771:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000af0fd3c8 17ffffc0001001 3:0 ffff9a78909bae80 120 lru 00000080:00200000:0.0:1787827241.396774:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f6ee5503 120: 1< 0 - - > 2< 491520 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396776:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007e44569c 00000080:00200000:0.0:1787827241.396777:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 121. 00000080:00200000:0.0:1787827241.396779:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 121 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396787:0:48385:0:(rw26.c:875:ll_write_end()) pos 495616, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396790:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000073d99898[3 00000000bdb38050 2 1 00000000746d8de4 0 0 121] 00000080:00200000:0.0:1787827241.396792:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006d666679 17ffffc0001001 3:0 ffff9a78909ba488 121 lru 00000080:00200000:0.0:1787827241.396796:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bd59441f 121: 1< 0 - - > 2< 495616 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396797:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000073d99898 00000080:00200000:0.0:1787827241.396798:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 122. 00000080:00200000:0.0:1787827241.396799:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 122 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396808:0:48385:0:(rw26.c:875:ll_write_end()) pos 499712, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396810:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000640a8efc[3 00000000bdb38050 2 1 00000000746d8de4 0 0 122] 00000080:00200000:0.0:1787827241.396812:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000018a50b72 17ffffc0001001 3:0 ffff9a78909babc8 122 lru 00000080:00200000:0.0:1787827241.396815:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000060a9a6b7 122: 1< 0 - - > 2< 499712 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396816:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000640a8efc 00000080:00200000:0.0:1787827241.396817:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 123. 00000080:00200000:0.0:1787827241.396819:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 123 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396827:0:48385:0:(rw26.c:875:ll_write_end()) pos 503808, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396844:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000005326f81[3 00000000bdb38050 2 1 00000000746d8de4 0 0 123] 00000080:00200000:0.0:1787827241.396848:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009b8261c7 17ffffc0001001 3:0 ffff9a78909ba570 123 lru 00000080:00200000:0.0:1787827241.396855:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000058662c7a 123: 1< 0 - - > 2< 503808 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396858:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000005326f81 00000080:00200000:0.0:1787827241.396860:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 124. 00000080:00200000:0.0:1787827241.396863:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 124 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396881:0:48385:0:(rw26.c:875:ll_write_end()) pos 507904, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396887:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001cf22622[3 00000000bdb38050 2 1 00000000746d8de4 0 0 124] 00000080:00200000:0.0:1787827241.396891:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000200fd790 17ffffc0001001 3:0 ffff9a78909ba828 124 lru 00000080:00200000:0.0:1787827241.396898:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008877649c 124: 1< 0 - - > 2< 507904 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396900:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001cf22622 00000080:00200000:0.0:1787827241.396902:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 125. 00000080:00200000:0.0:1787827241.396904:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 125 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396919:0:48385:0:(rw26.c:875:ll_write_end()) pos 512000, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396923:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000303ad32c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 125] 00000080:00200000:0.0:1787827241.396925:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f66626ff 17ffffc0001001 3:0 ffff9a78909ba000 125 lru 00000080:00200000:0.0:1787827241.396929:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007efbfdff 125: 1< 0 - - > 2< 512000 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396930:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000303ad32c 00000080:00200000:0.0:1787827241.396931:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 126. 00000080:00200000:0.0:1787827241.396933:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 126 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396945:0:48385:0:(rw26.c:875:ll_write_end()) pos 516096, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396947:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ec676ef0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 126] 00000080:00200000:0.0:1787827241.396949:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e6d52907 17ffffc0001001 3:0 ffff9a78909bacb0 126 lru 00000080:00200000:0.0:1787827241.396953:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000029684e6e 126: 1< 0 - - > 2< 516096 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396955:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ec676ef0 00000080:00200000:0.0:1787827241.396955:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 127. 00000080:00200000:0.0:1787827241.396957:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 127 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396965:0:48385:0:(rw26.c:875:ll_write_end()) pos 520192, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396967:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002996e515[3 00000000bdb38050 2 1 00000000746d8de4 0 0 127] 00000080:00200000:0.0:1787827241.396969:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a4598fae 17ffffc0001001 3:0 ffff9a78909baae0 127 lru 00000080:00200000:0.0:1787827241.396972:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fbdd9048 127: 1< 0 - - > 2< 520192 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396974:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002996e515 00000080:00200000:0.0:1787827241.396975:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 128. 00000080:00200000:0.0:1787827241.396976:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 128 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.396987:0:48385:0:(rw26.c:875:ll_write_end()) pos 524288, len 4096, copied 4096 00000080:00200000:0.0:1787827241.396990:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e1ae23e4[3 00000000bdb38050 2 1 00000000746d8de4 0 0 128] 00000080:00200000:0.0:1787827241.396992:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000047ea9f08 17ffffc0001001 3:0 ffff9a78909ba658 128 lru 00000080:00200000:0.0:1787827241.396996:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006ba1c21e 128: 1< 0 - - > 2< 524288 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.396998:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e1ae23e4 00000080:00200000:0.0:1787827241.396998:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 129. 00000080:00200000:0.0:1787827241.396999:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 129 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397007:0:48385:0:(rw26.c:875:ll_write_end()) pos 528384, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397010:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000db8b6308[3 00000000bdb38050 2 1 00000000746d8de4 0 0 129] 00000080:00200000:0.0:1787827241.397012:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005500d008 17ffffc0001001 3:0 ffff9a78909ba9f8 129 lru 00000080:00200000:0.0:1787827241.397015:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003af7cccc 129: 1< 0 - - > 2< 528384 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397017:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000db8b6308 00000080:00200000:0.0:1787827241.397018:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 130. 00000080:00200000:0.0:1787827241.397019:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 130 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397030:0:48385:0:(rw26.c:875:ll_write_end()) pos 532480, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397033:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000847068e5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 130] 00000080:00200000:0.0:1787827241.397035:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005ef2967f 17ffffc0001021 2:0 ffff9a78909ba910 130 lru 00000080:00200000:0.0:1787827241.397039:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cfbc7ba6 130: 1< 0 - - > 2< 532480 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397040:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000847068e5 00000080:00200000:0.0:1787827241.397041:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 131. 00000080:00200000:0.0:1787827241.397042:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 131 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397050:0:48385:0:(rw26.c:875:ll_write_end()) pos 536576, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397052:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008d5d4769[3 00000000bdb38050 2 1 00000000746d8de4 0 0 131] 00000080:00200000:0.0:1787827241.397054:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009991039b 17ffffc0001001 3:0 ffff9a78909bad98 131 lru 00000080:00200000:0.0:1787827241.397058:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ad29a6e3 131: 1< 0 - - > 2< 536576 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397059:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008d5d4769 00000080:00200000:0.0:1787827241.397060:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 132. 00000080:00200000:0.0:1787827241.397061:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 132 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397069:0:48385:0:(rw26.c:875:ll_write_end()) pos 540672, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397072:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e93484c7[3 00000000bdb38050 2 1 00000000746d8de4 0 0 132] 00000080:00200000:0.0:1787827241.397074:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d818d086 17ffffc0001001 3:0 ffff9a78909ba3a0 132 lru 00000080:00200000:0.0:1787827241.397077:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b5b07e0b 132: 1< 0 - - > 2< 540672 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397078:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e93484c7 00000080:00200000:0.0:1787827241.397079:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 133. 00000080:00200000:0.0:1787827241.397080:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 133 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397094:0:48385:0:(rw26.c:875:ll_write_end()) pos 544768, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397098:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009c7a06b0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 133] 00000080:00200000:0.0:1787827241.397101:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000039dce397 17ffffc0001001 3:0 ffff9a78909ba0e8 133 lru 00000080:00200000:0.0:1787827241.397106:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c183bff5 133: 1< 0 - - > 2< 544768 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397108:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009c7a06b0 00000080:00200000:0.0:1787827241.397110:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 134. 00000080:00200000:0.0:1787827241.397112:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 134 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397127:0:48385:0:(rw26.c:875:ll_write_end()) pos 548864, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397132:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a9fa5828[3 00000000bdb38050 2 1 00000000746d8de4 0 0 134] 00000080:00200000:0.0:1787827241.397135:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000038b8f9ec 17ffffc0001001 3:0 ffff9a78909ba2b8 134 lru 00000080:00200000:0.0:1787827241.397141:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007aae96fb 134: 1< 0 - - > 2< 548864 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397143:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a9fa5828 00000080:00200000:0.0:1787827241.397144:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 135. 00000080:00200000:0.0:1787827241.397146:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 135 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397157:0:48385:0:(rw26.c:875:ll_write_end()) pos 552960, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397160:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008a5f801c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 135] 00000080:00200000:0.0:1787827241.397162:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000433994f7 17ffffc0001001 3:0 ffff9a78909ba1d0 135 lru 00000080:00200000:0.0:1787827241.397166:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007bcd73e8 135: 1< 0 - - > 2< 552960 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397168:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008a5f801c 00000080:00200000:0.0:1787827241.397168:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 136. 00000080:00200000:0.0:1787827241.397169:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 136 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397178:0:48385:0:(rw26.c:875:ll_write_end()) pos 557056, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397180:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000078e7104e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 136] 00000080:00200000:0.0:1787827241.397182:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002f667e45 17ffffc0001001 3:0 ffff9a78909ba740 136 lru 00000080:00200000:0.0:1787827241.397185:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006dbf40d1 136: 1< 0 - - > 2< 557056 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397187:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000078e7104e 00000080:00200000:0.0:1787827241.397187:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 137. 00000080:00200000:0.0:1787827241.397189:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 137 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397204:0:48385:0:(rw26.c:875:ll_write_end()) pos 561152, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397207:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004f89c4d0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 137] 00000080:00200000:0.0:1787827241.397209:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f064ebdf 17ffffc0001001 3:0 ffff9a7884a38740 137 lru 00000080:00200000:0.0:1787827241.397213:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bfa6ac3f 137: 1< 0 - - > 2< 561152 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397214:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004f89c4d0 00000080:00200000:0.0:1787827241.397215:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 138. 00000080:00200000:0.0:1787827241.397216:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 138 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397224:0:48385:0:(rw26.c:875:ll_write_end()) pos 565248, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397227:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000066c7af84[3 00000000bdb38050 2 1 00000000746d8de4 0 0 138] 00000080:00200000:0.0:1787827241.397228:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000078eceeeb 17ffffc0001001 3:0 ffff9a7884a38e80 138 lru 00000080:00200000:0.0:1787827241.397232:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000019b1c2e0 138: 1< 0 - - > 2< 565248 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397234:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000066c7af84 00000080:00200000:0.0:1787827241.397234:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 139. 00000080:00200000:0.0:1787827241.397236:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 139 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397244:0:48385:0:(rw26.c:875:ll_write_end()) pos 569344, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397247:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008007028f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 139] 00000080:00200000:0.0:1787827241.397249:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005471cc15 17ffffc0001001 3:0 ffff9a7884a38488 139 lru 00000080:00200000:0.0:1787827241.397253:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e61ff07a 139: 1< 0 - - > 2< 569344 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397254:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008007028f 00000080:00200000:0.0:1787827241.397255:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 140. 00000080:00200000:0.0:1787827241.397256:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 140 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397266:0:48385:0:(rw26.c:875:ll_write_end()) pos 573440, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397268:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c089a750[3 00000000bdb38050 2 1 00000000746d8de4 0 0 140] 00000080:00200000:0.0:1787827241.397270:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000796259f0 17ffffc0001001 3:0 ffff9a7884a38bc8 140 lru 00000080:00200000:0.0:1787827241.397273:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002263070e 140: 1< 0 - - > 2< 573440 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397274:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c089a750 00000080:00200000:0.0:1787827241.397275:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 141. 00000080:00200000:0.0:1787827241.397276:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 141 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397285:0:48385:0:(rw26.c:875:ll_write_end()) pos 577536, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397288:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008a00e6ce[3 00000000bdb38050 2 1 00000000746d8de4 0 0 141] 00000080:00200000:0.0:1787827241.397290:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a61bcc76 17ffffc0001001 3:0 ffff9a7884a38570 141 lru 00000080:00200000:0.0:1787827241.397293:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000af26f94b 141: 1< 0 - - > 2< 577536 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397294:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008a00e6ce 00000080:00200000:0.0:1787827241.397295:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 142. 00000080:00200000:0.0:1787827241.397296:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 142 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397304:0:48385:0:(rw26.c:875:ll_write_end()) pos 581632, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397307:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d0987cec[3 00000000bdb38050 2 1 00000000746d8de4 0 0 142] 00000080:00200000:0.0:1787827241.397309:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000035c5fb4c 17ffffc0001001 3:0 ffff9a7884a38828 142 lru 00000080:00200000:0.0:1787827241.397312:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a5edbf8f 142: 1< 0 - - > 2< 581632 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397313:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d0987cec 00000080:00200000:0.0:1787827241.397314:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 143. 00000080:00200000:0.0:1787827241.397315:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 143 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397324:0:48385:0:(rw26.c:875:ll_write_end()) pos 585728, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397326:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e76e1bed[3 00000000bdb38050 2 1 00000000746d8de4 0 0 143] 00000080:00200000:0.0:1787827241.397328:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000091dda1b3 17ffffc0001001 3:0 ffff9a7884a38000 143 lru 00000080:00200000:0.0:1787827241.397332:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a539dc4a 143: 1< 0 - - > 2< 585728 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397334:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e76e1bed 00000080:00200000:0.0:1787827241.397334:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 144. 00000080:00200000:0.0:1787827241.397335:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 144 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397343:0:48385:0:(rw26.c:875:ll_write_end()) pos 589824, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397346:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000170dd1d8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 144] 00000080:00200000:0.0:1787827241.397348:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f3ecc8d5 17ffffc0001001 3:0 ffff9a7884a38cb0 144 lru 00000080:00200000:0.0:1787827241.397351:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000785dc6f0 144: 1< 0 - - > 2< 589824 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397352:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000170dd1d8 00000080:00200000:0.0:1787827241.397353:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 145. 00000080:00200000:0.0:1787827241.397354:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 145 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397365:0:48385:0:(rw26.c:875:ll_write_end()) pos 593920, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397367:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009d25265a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 145] 00000080:00200000:0.0:1787827241.397369:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000699d30fd 17ffffc0001021 2:0 ffff9a7884a38ae0 145 lru 00000080:00200000:0.0:1787827241.397372:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000071dc9d63 145: 1< 0 - - > 2< 593920 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397374:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009d25265a 00000080:00200000:0.0:1787827241.397375:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 146. 00000080:00200000:0.0:1787827241.397376:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 146 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397385:0:48385:0:(rw26.c:875:ll_write_end()) pos 598016, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397387:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005ae112bd[3 00000000bdb38050 2 1 00000000746d8de4 0 0 146] 00000080:00200000:0.0:1787827241.397389:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000034f64940 17ffffc0001001 3:0 ffff9a7884a38658 146 lru 00000080:00200000:0.0:1787827241.397393:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c5c88a61 146: 1< 0 - - > 2< 598016 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397395:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005ae112bd 00000080:00200000:0.0:1787827241.397395:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 147. 00000080:00200000:0.0:1787827241.397396:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 147 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397406:0:48385:0:(rw26.c:875:ll_write_end()) pos 602112, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397409:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009ec14a50[3 00000000bdb38050 2 1 00000000746d8de4 0 0 147] 00000080:00200000:0.0:1787827241.397410:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000debf4097 17ffffc0001001 3:0 ffff9a7884a389f8 147 lru 00000080:00200000:0.0:1787827241.397414:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000323eab85 147: 1< 0 - - > 2< 602112 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397416:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009ec14a50 00000080:00200000:0.0:1787827241.397417:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 148. 00000080:00200000:0.0:1787827241.397418:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 148 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397426:0:48385:0:(rw26.c:875:ll_write_end()) pos 606208, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397429:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002afbbd4b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 148] 00000080:00200000:0.0:1787827241.397431:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000079996594 17ffffc0001001 3:0 ffff9a7884a38910 148 lru 00000080:00200000:0.0:1787827241.397435:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ea11ca5a 148: 1< 0 - - > 2< 606208 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397436:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002afbbd4b 00000080:00200000:0.0:1787827241.397437:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 149. 00000080:00200000:0.0:1787827241.397438:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 149 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397446:0:48385:0:(rw26.c:875:ll_write_end()) pos 610304, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397448:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005af4f753[3 00000000bdb38050 2 1 00000000746d8de4 0 0 149] 00000080:00200000:0.0:1787827241.397450:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ea8b4ec3 17ffffc0001001 3:0 ffff9a7884a38d98 149 lru 00000080:00200000:0.0:1787827241.397454:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004ff40017 149: 1< 0 - - > 2< 610304 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397455:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005af4f753 00000080:00200000:0.0:1787827241.397455:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 150. 00000080:00200000:0.0:1787827241.397456:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 150 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397465:0:48385:0:(rw26.c:875:ll_write_end()) pos 614400, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397467:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003d5fb949[3 00000000bdb38050 2 1 00000000746d8de4 0 0 150] 00000080:00200000:0.0:1787827241.397469:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000356e9bce 17ffffc0001001 3:0 ffff9a7884a383a0 150 lru 00000080:00200000:0.0:1787827241.397472:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001025c52c 150: 1< 0 - - > 2< 614400 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397474:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003d5fb949 00000080:00200000:0.0:1787827241.397474:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 151. 00000080:00200000:0.0:1787827241.397475:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 151 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397483:0:48385:0:(rw26.c:875:ll_write_end()) pos 618496, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397486:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000023357746[3 00000000bdb38050 2 1 00000000746d8de4 0 0 151] 00000080:00200000:0.0:1787827241.397487:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000013313240 17ffffc0001001 3:0 ffff9a7884a380e8 151 lru 00000080:00200000:0.0:1787827241.397491:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ee379a6a 151: 1< 0 - - > 2< 618496 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397492:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000023357746 00000080:00200000:0.0:1787827241.397493:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 152. 00000080:00200000:0.0:1787827241.397494:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 152 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397516:0:48385:0:(rw26.c:875:ll_write_end()) pos 622592, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397520:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a89d1b7a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 152] 00000080:00200000:0.0:1787827241.397521:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000de2318fe 17ffffc0001001 3:0 ffff9a7884a382b8 152 lru 00000080:00200000:0.0:1787827241.397525:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009291acd8 152: 1< 0 - - > 2< 622592 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397527:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a89d1b7a 00000080:00200000:0.0:1787827241.397527:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 153. 00000080:00200000:0.0:1787827241.397529:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 153 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397540:0:48385:0:(rw26.c:875:ll_write_end()) pos 626688, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397543:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006c4bbef4[3 00000000bdb38050 2 1 00000000746d8de4 0 0 153] 00000080:00200000:0.0:1787827241.397545:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d362afbb 17ffffc0001001 3:0 ffff9a7884a381d0 153 lru 00000080:00200000:0.0:1787827241.397549:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000443fb72f 153: 1< 0 - - > 2< 626688 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397551:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006c4bbef4 00000080:00200000:0.0:1787827241.397551:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 154. 00000080:00200000:0.0:1787827241.397552:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 154 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397569:0:48385:0:(rw26.c:875:ll_write_end()) pos 630784, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397572:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000036f84cc2[3 00000000bdb38050 2 1 00000000746d8de4 0 0 154] 00000080:00200000:0.0:1787827241.397573:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000069b2650e 17ffffc0001001 3:0 ffff9a78a658c740 154 lru 00000080:00200000:0.0:1787827241.397577:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000075fac176 154: 1< 0 - - > 2< 630784 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397579:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000036f84cc2 00000080:00200000:0.0:1787827241.397579:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 155. 00000080:00200000:0.0:1787827241.397580:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 155 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397589:0:48385:0:(rw26.c:875:ll_write_end()) pos 634880, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397591:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000af2516a9[3 00000000bdb38050 2 1 00000000746d8de4 0 0 155] 00000080:00200000:0.0:1787827241.397593:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000037fff255 17ffffc0001001 3:0 ffff9a78a658ce80 155 lru 00000080:00200000:0.0:1787827241.397597:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005fa2f6f6 155: 1< 0 - - > 2< 634880 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397599:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000af2516a9 00000080:00200000:0.0:1787827241.397600:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 156. 00000080:00200000:0.0:1787827241.397601:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 156 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397609:0:48385:0:(rw26.c:875:ll_write_end()) pos 638976, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397611:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c1e4389e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 156] 00000080:00200000:0.0:1787827241.397613:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cae3d3bc 17ffffc0001001 3:0 ffff9a78a658c488 156 lru 00000080:00200000:0.0:1787827241.397616:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000094d866df 156: 1< 0 - - > 2< 638976 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397618:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c1e4389e 00000080:00200000:0.0:1787827241.397619:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 157. 00000080:00200000:0.0:1787827241.397621:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 157 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397630:0:48385:0:(rw26.c:875:ll_write_end()) pos 643072, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397632:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c3542e22[3 00000000bdb38050 2 1 00000000746d8de4 0 0 157] 00000080:00200000:0.0:1787827241.397634:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001f4d74ec 17ffffc0001001 3:0 ffff9a78a658cbc8 157 lru 00000080:00200000:0.0:1787827241.397638:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e43a384a 157: 1< 0 - - > 2< 643072 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397640:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c3542e22 00000080:00200000:0.0:1787827241.397640:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 158. 00000080:00200000:0.0:1787827241.397641:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 158 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397650:0:48385:0:(rw26.c:875:ll_write_end()) pos 647168, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397652:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004b50a720[3 00000000bdb38050 2 1 00000000746d8de4 0 0 158] 00000080:00200000:0.0:1787827241.397654:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000094dce07 17ffffc0001001 3:0 ffff9a78a658c570 158 lru 00000080:00200000:0.0:1787827241.397658:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006efa9eba 158: 1< 0 - - > 2< 647168 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397659:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004b50a720 00000080:00200000:0.0:1787827241.397660:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 159. 00000080:00200000:0.0:1787827241.397661:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 159 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397669:0:48385:0:(rw26.c:875:ll_write_end()) pos 651264, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397672:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009a12027a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 159] 00000080:00200000:0.0:1787827241.397674:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000043130c9f 17ffffc0001001 3:0 ffff9a78a658c828 159 lru 00000080:00200000:0.0:1787827241.397678:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000060de6bee 159: 1< 0 - - > 2< 651264 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397680:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009a12027a 00000080:00200000:0.0:1787827241.397680:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 160. 00000080:00200000:0.0:1787827241.397681:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 160 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397692:0:48385:0:(rw26.c:875:ll_write_end()) pos 655360, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397695:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b61e62ed[3 00000000bdb38050 2 1 00000000746d8de4 0 0 160] 00000080:00200000:0.0:1787827241.397697:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000b0979ed 17ffffc0001021 2:0 ffff9a78a658c000 160 lru 00000080:00200000:0.0:1787827241.397700:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c8d24a3c 160: 1< 0 - - > 2< 655360 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397701:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b61e62ed 00000080:00200000:0.0:1787827241.397702:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 161. 00000080:00200000:0.0:1787827241.397703:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 161 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397714:0:48385:0:(rw26.c:875:ll_write_end()) pos 659456, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397716:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c4abdb4a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 161] 00000080:00200000:0.0:1787827241.397718:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ea22a014 17ffffc0001001 3:0 ffff9a78a658ccb0 161 lru 00000080:00200000:0.0:1787827241.397723:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f0e7fd68 161: 1< 0 - - > 2< 659456 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397724:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c4abdb4a 00000080:00200000:0.0:1787827241.397725:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 162. 00000080:00200000:0.0:1787827241.397726:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 162 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397735:0:48385:0:(rw26.c:875:ll_write_end()) pos 663552, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397737:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000060f858da[3 00000000bdb38050 2 1 00000000746d8de4 0 0 162] 00000080:00200000:0.0:1787827241.397739:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000443c23a2 17ffffc0001001 3:0 ffff9a78a658cae0 162 lru 00000080:00200000:0.0:1787827241.397742:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002083ea38 162: 1< 0 - - > 2< 663552 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397744:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000060f858da 00000080:00200000:0.0:1787827241.397744:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 163. 00000080:00200000:0.0:1787827241.397746:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 163 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397754:0:48385:0:(rw26.c:875:ll_write_end()) pos 667648, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397757:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000334a5d60[3 00000000bdb38050 2 1 00000000746d8de4 0 0 163] 00000080:00200000:0.0:1787827241.397759:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f5e5fa5d 17ffffc0001001 3:0 ffff9a78a658c658 163 lru 00000080:00200000:0.0:1787827241.397762:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000061d8f0a0 163: 1< 0 - - > 2< 667648 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397764:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000334a5d60 00000080:00200000:0.0:1787827241.397765:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 164. 00000080:00200000:0.0:1787827241.397766:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 164 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397774:0:48385:0:(rw26.c:875:ll_write_end()) pos 671744, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397777:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000039496f90[3 00000000bdb38050 2 1 00000000746d8de4 0 0 164] 00000080:00200000:0.0:1787827241.397778:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ab565c11 17ffffc0001001 3:0 ffff9a78a658c9f8 164 lru 00000080:00200000:0.0:1787827241.397782:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002d849c7b 164: 1< 0 - - > 2< 671744 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397783:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000039496f90 00000080:00200000:0.0:1787827241.397783:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 165. 00000080:00200000:0.0:1787827241.397784:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 165 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397793:0:48385:0:(rw26.c:875:ll_write_end()) pos 675840, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397795:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000331de893[3 00000000bdb38050 2 1 00000000746d8de4 0 0 165] 00000080:00200000:0.0:1787827241.397797:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000078de4590 17ffffc0001001 3:0 ffff9a78a658c910 165 lru 00000080:00200000:0.0:1787827241.397800:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000061fbd4 165: 1< 0 - - > 2< 675840 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397802:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000331de893 00000080:00200000:0.0:1787827241.397803:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 166. 00000080:00200000:0.0:1787827241.397804:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 166 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397812:0:48385:0:(rw26.c:875:ll_write_end()) pos 679936, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397815:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bc335583[3 00000000bdb38050 2 1 00000000746d8de4 0 0 166] 00000080:00200000:0.0:1787827241.397816:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000006a0bc75 17ffffc0001001 3:0 ffff9a78a658cd98 166 lru 00000080:00200000:0.0:1787827241.397820:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009514cb9d 166: 1< 0 - - > 2< 679936 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397821:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bc335583 00000080:00200000:0.0:1787827241.397822:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 167. 00000080:00200000:0.0:1787827241.397823:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 167 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397850:0:48385:0:(rw26.c:875:ll_write_end()) pos 684032, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397856:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007caf728e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 167] 00000080:00200000:0.0:1787827241.397860:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006345e64d 17ffffc0001001 3:0 ffff9a78a658c3a0 167 lru 00000080:00200000:0.0:1787827241.397867:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000be257c50 167: 1< 0 - - > 2< 684032 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397870:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007caf728e 00000080:00200000:0.0:1787827241.397872:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 168. 00000080:00200000:0.0:1787827241.397874:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 168 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397894:0:48385:0:(rw26.c:875:ll_write_end()) pos 688128, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397899:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000046074c51[3 00000000bdb38050 2 1 00000000746d8de4 0 0 168] 00000080:00200000:0.0:1787827241.397903:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a1c1aafc 17ffffc0001001 3:0 ffff9a78a658c0e8 168 lru 00000080:00200000:0.0:1787827241.397909:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008a078642 168: 1< 0 - - > 2< 688128 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397912:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000046074c51 00000080:00200000:0.0:1787827241.397913:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 169. 00000080:00200000:0.0:1787827241.397914:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 169 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397926:0:48385:0:(rw26.c:875:ll_write_end()) pos 692224, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397929:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000045aceabd[3 00000000bdb38050 2 1 00000000746d8de4 0 0 169] 00000080:00200000:0.0:1787827241.397931:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006edafcc8 17ffffc0001001 3:0 ffff9a78a658c2b8 169 lru 00000080:00200000:0.0:1787827241.397935:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000000ad6cf0 169: 1< 0 - - > 2< 692224 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397936:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000045aceabd 00000080:00200000:0.0:1787827241.397937:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 170. 00000080:00200000:0.0:1787827241.397938:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 170 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397947:0:48385:0:(rw26.c:875:ll_write_end()) pos 696320, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397949:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004001abe0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 170] 00000080:00200000:0.0:1787827241.397951:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000052fad68 17ffffc0001001 3:0 ffff9a78a658c1d0 170 lru 00000080:00200000:0.0:1787827241.397954:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c7e69e05 170: 1< 0 - - > 2< 696320 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397956:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004001abe0 00000080:00200000:0.0:1787827241.397957:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 171. 00000080:00200000:0.0:1787827241.397958:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 171 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397974:0:48385:0:(rw26.c:875:ll_write_end()) pos 700416, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397976:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009ff43550[3 00000000bdb38050 2 1 00000000746d8de4 0 0 171] 00000080:00200000:0.0:1787827241.397978:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003d4c4062 17ffffc0001001 3:0 ffff9a78a65c5828 171 lru 00000080:00200000:0.0:1787827241.397982:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000034936952 171: 1< 0 - - > 2< 700416 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.397984:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009ff43550 00000080:00200000:0.0:1787827241.397984:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 172. 00000080:00200000:0.0:1787827241.397985:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 172 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.397993:0:48385:0:(rw26.c:875:ll_write_end()) pos 704512, len 4096, copied 4096 00000080:00200000:0.0:1787827241.397996:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009326db07[3 00000000bdb38050 2 1 00000000746d8de4 0 0 172] 00000080:00200000:0.0:1787827241.397997:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000054bab3e9 17ffffc0001001 3:0 ffff9a78a65c5000 172 lru 00000080:00200000:0.0:1787827241.398001:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000a578858 172: 1< 0 - - > 2< 704512 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398002:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009326db07 00000080:00200000:0.0:1787827241.398003:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 173. 00000080:00200000:0.0:1787827241.398004:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 173 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398013:0:48385:0:(rw26.c:875:ll_write_end()) pos 708608, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398015:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003e7a0477[3 00000000bdb38050 2 1 00000000746d8de4 0 0 173] 00000080:00200000:0.0:1787827241.398017:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e2b7869b 17ffffc0001001 3:0 ffff9a78a65c5cb0 173 lru 00000080:00200000:0.0:1787827241.398021:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000042f7596a 173: 1< 0 - - > 2< 708608 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398023:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003e7a0477 00000080:00200000:0.0:1787827241.398023:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 174. 00000080:00200000:0.0:1787827241.398025:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 174 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398034:0:48385:0:(rw26.c:875:ll_write_end()) pos 712704, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398036:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000471c78d7[3 00000000bdb38050 2 1 00000000746d8de4 0 0 174] 00000080:00200000:0.0:1787827241.398038:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a63c42e5 17ffffc0001001 3:0 ffff9a78a65c5ae0 174 lru 00000080:00200000:0.0:1787827241.398042:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000365e0eae 174: 1< 0 - - > 2< 712704 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398044:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000471c78d7 00000080:00200000:0.0:1787827241.398045:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 175. 00000080:00200000:0.0:1787827241.398046:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 175 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398058:0:48385:0:(rw26.c:875:ll_write_end()) pos 716800, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398061:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000005035bf6[3 00000000bdb38050 2 1 00000000746d8de4 0 0 175] 00000080:00200000:0.0:1787827241.398063:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000949cbc13 17ffffc0001021 2:0 ffff9a78a65c5658 175 lru 00000080:00200000:0.0:1787827241.398067:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002676e82e 175: 1< 0 - - > 2< 716800 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398068:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000005035bf6 00000080:00200000:0.0:1787827241.398069:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 176. 00000080:00200000:0.0:1787827241.398070:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 176 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398078:0:48385:0:(rw26.c:875:ll_write_end()) pos 720896, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398081:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000024b3d297[3 00000000bdb38050 2 1 00000000746d8de4 0 0 176] 00000080:00200000:0.0:1787827241.398083:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002da95b20 17ffffc0001001 3:0 ffff9a78a65c59f8 176 lru 00000080:00200000:0.0:1787827241.398089:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ee3ac89e 176: 1< 0 - - > 2< 720896 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398091:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000024b3d297 00000080:00200000:0.0:1787827241.398092:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 177. 00000080:00200000:0.0:1787827241.398094:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 177 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398108:0:48385:0:(rw26.c:875:ll_write_end()) pos 724992, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398113:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000959bdf60[3 00000000bdb38050 2 1 00000000746d8de4 0 0 177] 00000080:00200000:0.0:1787827241.398117:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a5f3b566 17ffffc0001001 3:0 ffff9a78a65c5910 177 lru 00000080:00200000:0.0:1787827241.398123:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007eb54682 177: 1< 0 - - > 2< 724992 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398126:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000959bdf60 00000080:00200000:0.0:1787827241.398127:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 178. 00000080:00200000:0.0:1787827241.398129:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 178 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398143:0:48385:0:(rw26.c:875:ll_write_end()) pos 729088, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398146:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b55d70e9[3 00000000bdb38050 2 1 00000000746d8de4 0 0 178] 00000080:00200000:0.0:1787827241.398148:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009bfecb4a 17ffffc0001001 3:0 ffff9a78a65c5d98 178 lru 00000080:00200000:0.0:1787827241.398152:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ef2439ea 178: 1< 0 - - > 2< 729088 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398154:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b55d70e9 00000080:00200000:0.0:1787827241.398155:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 179. 00000080:00200000:0.0:1787827241.398156:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 179 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398165:0:48385:0:(rw26.c:875:ll_write_end()) pos 733184, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398167:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000099dd960f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 179] 00000080:00200000:0.0:1787827241.398170:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009f7b9966 17ffffc0001001 3:0 ffff9a78a65c53a0 179 lru 00000080:00200000:0.0:1787827241.398173:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000001adc4a3 179: 1< 0 - - > 2< 733184 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398175:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000099dd960f 00000080:00200000:0.0:1787827241.398176:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 180. 00000080:00200000:0.0:1787827241.398177:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 180 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398185:0:48385:0:(rw26.c:875:ll_write_end()) pos 737280, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398187:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007cd2617c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 180] 00000080:00200000:0.0:1787827241.398189:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006f2b19b6 17ffffc0001001 3:0 ffff9a78a65c50e8 180 lru 00000080:00200000:0.0:1787827241.398192:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c83e94bc 180: 1< 0 - - > 2< 737280 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398194:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007cd2617c 00000080:00200000:0.0:1787827241.398195:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 181. 00000080:00200000:0.0:1787827241.398196:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 181 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398204:0:48385:0:(rw26.c:875:ll_write_end()) pos 741376, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398207:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001e3043a8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 181] 00000080:00200000:0.0:1787827241.398209:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000171b8de8 17ffffc0001001 3:0 ffff9a78a65c52b8 181 lru 00000080:00200000:0.0:1787827241.398212:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008bd25425 181: 1< 0 - - > 2< 741376 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398214:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001e3043a8 00000080:00200000:0.0:1787827241.398215:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 182. 00000080:00200000:0.0:1787827241.398216:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 182 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398226:0:48385:0:(rw26.c:875:ll_write_end()) pos 745472, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398229:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000013866f31[3 00000000bdb38050 2 1 00000000746d8de4 0 0 182] 00000080:00200000:0.0:1787827241.398231:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ed908867 17ffffc0001001 3:0 ffff9a78a65c51d0 182 lru 00000080:00200000:0.0:1787827241.398234:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004637163c 182: 1< 0 - - > 2< 745472 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398236:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000013866f31 00000080:00200000:0.0:1787827241.398237:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 183. 00000080:00200000:0.0:1787827241.398238:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 183 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398246:0:48385:0:(rw26.c:875:ll_write_end()) pos 749568, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398248:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000e2caac2[3 00000000bdb38050 2 1 00000000746d8de4 0 0 183] 00000080:00200000:0.0:1787827241.398250:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000016e08f2f 17ffffc0001001 3:0 ffff9a78a65c5740 183 lru 00000080:00200000:0.0:1787827241.398254:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003cb73676 183: 1< 0 - - > 2< 749568 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398256:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000e2caac2 00000080:00200000:0.0:1787827241.398257:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 184. 00000080:00200000:0.0:1787827241.398258:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 184 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398266:0:48385:0:(rw26.c:875:ll_write_end()) pos 753664, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398269:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000010be04a4[3 00000000bdb38050 2 1 00000000746d8de4 0 0 184] 00000080:00200000:0.0:1787827241.398271:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000097934503 17ffffc0001001 3:0 ffff9a78a65c5e80 184 lru 00000080:00200000:0.0:1787827241.398275:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000307d197 184: 1< 0 - - > 2< 753664 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398277:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000010be04a4 00000080:00200000:0.0:1787827241.398277:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 185. 00000080:00200000:0.0:1787827241.398278:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 185 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398287:0:48385:0:(rw26.c:875:ll_write_end()) pos 757760, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398289:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000030022748[3 00000000bdb38050 2 1 00000000746d8de4 0 0 185] 00000080:00200000:0.0:1787827241.398291:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b1f610a1 17ffffc0001001 3:0 ffff9a78a65c5488 185 lru 00000080:00200000:0.0:1787827241.398294:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000594b4023 185: 1< 0 - - > 2< 757760 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398295:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000030022748 00000080:00200000:0.0:1787827241.398296:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 186. 00000080:00200000:0.0:1787827241.398297:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 186 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398305:0:48385:0:(rw26.c:875:ll_write_end()) pos 761856, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398308:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000083b1ee2f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 186] 00000080:00200000:0.0:1787827241.398309:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f0069e46 17ffffc0001001 3:0 ffff9a78a65c5bc8 186 lru 00000080:00200000:0.0:1787827241.398313:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b8fdf6f6 186: 1< 0 - - > 2< 761856 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398314:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000083b1ee2f 00000080:00200000:0.0:1787827241.398315:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 187. 00000080:00200000:0.0:1787827241.398316:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 187 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398324:0:48385:0:(rw26.c:875:ll_write_end()) pos 765952, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398326:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c3645b8b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 187] 00000080:00200000:0.0:1787827241.398328:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008f797261 17ffffc0001001 3:0 ffff9a78a65c5570 187 lru 00000080:00200000:0.0:1787827241.398331:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000036f525c0 187: 1< 0 - - > 2< 765952 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398333:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c3645b8b 00000080:00200000:0.0:1787827241.398334:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 188. 00000080:00200000:0.0:1787827241.398335:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 188 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398350:0:48385:0:(rw26.c:875:ll_write_end()) pos 770048, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398353:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008096cf35[3 00000000bdb38050 2 1 00000000746d8de4 0 0 188] 00000080:00200000:0.0:1787827241.398355:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f2116a6d 17ffffc0001001 3:0 ffff9a7886958570 188 lru 00000080:00200000:0.0:1787827241.398358:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009988ff53 188: 1< 0 - - > 2< 770048 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398360:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008096cf35 00000080:00200000:0.0:1787827241.398361:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 189. 00000080:00200000:0.0:1787827241.398362:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 189 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398373:0:48385:0:(rw26.c:875:ll_write_end()) pos 774144, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398375:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b524c29a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 189] 00000080:00200000:0.0:1787827241.398377:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c6d9b742 17ffffc0001001 3:0 ffff9a7886958828 189 lru 00000080:00200000:0.0:1787827241.398381:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001b737226 189: 1< 0 - - > 2< 774144 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398382:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b524c29a 00000080:00200000:0.0:1787827241.398383:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 190. 00000080:00200000:0.0:1787827241.398384:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 190 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398396:0:48385:0:(rw26.c:875:ll_write_end()) pos 778240, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398398:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007418d0c4[3 00000000bdb38050 2 1 00000000746d8de4 0 0 190] 00000080:00200000:0.0:1787827241.398400:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000842ef5f1 17ffffc0001021 2:0 ffff9a7886958000 190 lru 00000080:00200000:0.0:1787827241.398404:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002f7c0fa3 190: 1< 0 - - > 2< 778240 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398406:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007418d0c4 00000080:00200000:0.0:1787827241.398406:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 191. 00000080:00200000:0.0:1787827241.398408:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 191 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398416:0:48385:0:(rw26.c:875:ll_write_end()) pos 782336, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398418:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a7b3f1db[3 00000000bdb38050 2 1 00000000746d8de4 0 0 191] 00000080:00200000:0.0:1787827241.398420:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000293f2708 17ffffc0001001 3:0 ffff9a7886958cb0 191 lru 00000080:00200000:0.0:1787827241.398424:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cd33a9b8 191: 1< 0 - - > 2< 782336 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398425:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a7b3f1db 00000080:00200000:0.0:1787827241.398426:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 192. 00000080:00200000:0.0:1787827241.398427:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 192 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398438:0:48385:0:(rw26.c:875:ll_write_end()) pos 786432, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398441:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001b129fff[3 00000000bdb38050 2 1 00000000746d8de4 0 0 192] 00000080:00200000:0.0:1787827241.398443:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006e97cada 17ffffc0001001 3:0 ffff9a7886958ae0 192 lru 00000080:00200000:0.0:1787827241.398447:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e2e71077 192: 1< 0 - - > 2< 786432 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398449:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001b129fff 00000080:00200000:0.0:1787827241.398450:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 193. 00000080:00200000:0.0:1787827241.398451:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 193 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398460:0:48385:0:(rw26.c:875:ll_write_end()) pos 790528, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398463:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004a772cef[3 00000000bdb38050 2 1 00000000746d8de4 0 0 193] 00000080:00200000:0.0:1787827241.398465:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cf6f4b9b 17ffffc0001001 3:0 ffff9a7886958658 193 lru 00000080:00200000:0.0:1787827241.398469:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000cecc274 193: 1< 0 - - > 2< 790528 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398470:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004a772cef 00000080:00200000:0.0:1787827241.398471:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 194. 00000080:00200000:0.0:1787827241.398472:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 194 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398481:0:48385:0:(rw26.c:875:ll_write_end()) pos 794624, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398483:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000023849dd7[3 00000000bdb38050 2 1 00000000746d8de4 0 0 194] 00000080:00200000:0.0:1787827241.398485:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e25cb03b 17ffffc0001001 3:0 ffff9a78869589f8 194 lru 00000080:00200000:0.0:1787827241.398488:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000017029f1 194: 1< 0 - - > 2< 794624 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398489:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000023849dd7 00000080:00200000:0.0:1787827241.398490:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 195. 00000080:00200000:0.0:1787827241.398491:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 195 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398500:0:48385:0:(rw26.c:875:ll_write_end()) pos 798720, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398517:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000984caf1a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 195] 00000080:00200000:0.0:1787827241.398519:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000014fdf302 17ffffc0001001 3:0 ffff9a7886958910 195 lru 00000080:00200000:0.0:1787827241.398523:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000082450f86 195: 1< 0 - - > 2< 798720 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398525:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000984caf1a 00000080:00200000:0.0:1787827241.398525:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 196. 00000080:00200000:0.0:1787827241.398527:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 196 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398540:0:48385:0:(rw26.c:875:ll_write_end()) pos 802816, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398543:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001a521d6e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 196] 00000080:00200000:0.0:1787827241.398544:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005c8b905f 17ffffc0001001 3:0 ffff9a7886958d98 196 lru 00000080:00200000:0.0:1787827241.398548:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007e6f794b 196: 1< 0 - - > 2< 802816 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398550:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001a521d6e 00000080:00200000:0.0:1787827241.398551:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 197. 00000080:00200000:0.0:1787827241.398552:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 197 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398561:0:48385:0:(rw26.c:875:ll_write_end()) pos 806912, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398564:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bb2d7480[3 00000000bdb38050 2 1 00000000746d8de4 0 0 197] 00000080:00200000:0.0:1787827241.398566:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bf04db6c 17ffffc0001001 3:0 ffff9a78869583a0 197 lru 00000080:00200000:0.0:1787827241.398570:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000922326c7 197: 1< 0 - - > 2< 806912 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398572:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bb2d7480 00000080:00200000:0.0:1787827241.398572:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 198. 00000080:00200000:0.0:1787827241.398573:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 198 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398582:0:48385:0:(rw26.c:875:ll_write_end()) pos 811008, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398584:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000058c53dd[3 00000000bdb38050 2 1 00000000746d8de4 0 0 198] 00000080:00200000:0.0:1787827241.398586:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000063929190 17ffffc0001001 3:0 ffff9a78869580e8 198 lru 00000080:00200000:0.0:1787827241.398589:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000014dfe6e7 198: 1< 0 - - > 2< 811008 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398591:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000058c53dd 00000080:00200000:0.0:1787827241.398592:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 199. 00000080:00200000:0.0:1787827241.398593:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 199 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398601:0:48385:0:(rw26.c:875:ll_write_end()) pos 815104, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398604:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fa463a25[3 00000000bdb38050 2 1 00000000746d8de4 0 0 199] 00000080:00200000:0.0:1787827241.398606:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e23c712d 17ffffc0001001 3:0 ffff9a78869582b8 199 lru 00000080:00200000:0.0:1787827241.398610:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007c06e717 199: 1< 0 - - > 2< 815104 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398612:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fa463a25 00000080:00200000:0.0:1787827241.398612:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 200. 00000080:00200000:0.0:1787827241.398614:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 200 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398622:0:48385:0:(rw26.c:875:ll_write_end()) pos 819200, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398624:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e0dcb034[3 00000000bdb38050 2 1 00000000746d8de4 0 0 200] 00000080:00200000:0.0:1787827241.398626:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000723a0b85 17ffffc0001001 3:0 ffff9a78869581d0 200 lru 00000080:00200000:0.0:1787827241.398630:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002a3cdf1c 200: 1< 0 - - > 2< 819200 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398632:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e0dcb034 00000080:00200000:0.0:1787827241.398632:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 201. 00000080:00200000:0.0:1787827241.398634:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 201 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398642:0:48385:0:(rw26.c:875:ll_write_end()) pos 823296, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398645:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003b67b28d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 201] 00000080:00200000:0.0:1787827241.398646:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004f3d497f 17ffffc0001001 3:0 ffff9a7886958740 201 lru 00000080:00200000:0.0:1787827241.398650:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007398de8d 201: 1< 0 - - > 2< 823296 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398653:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003b67b28d 00000080:00200000:0.0:1787827241.398654:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 202. 00000080:00200000:0.0:1787827241.398655:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 202 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398664:0:48385:0:(rw26.c:875:ll_write_end()) pos 827392, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398667:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b403cdf5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 202] 00000080:00200000:0.0:1787827241.398669:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005fbf8b92 17ffffc0001001 3:0 ffff9a7886958e80 202 lru 00000080:00200000:0.0:1787827241.398673:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000bdbfe66a 202: 1< 0 - - > 2< 827392 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398674:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b403cdf5 00000080:00200000:0.0:1787827241.398675:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 203. 00000080:00200000:0.0:1787827241.398676:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 203 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398685:0:48385:0:(rw26.c:875:ll_write_end()) pos 831488, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398688:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004442b40d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 203] 00000080:00200000:0.0:1787827241.398689:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008eab8ef9 17ffffc0001001 3:0 ffff9a7886958488 203 lru 00000080:00200000:0.0:1787827241.398693:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a8ac1a97 203: 1< 0 - - > 2< 831488 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398694:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004442b40d 00000080:00200000:0.0:1787827241.398695:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 204. 00000080:00200000:0.0:1787827241.398696:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 204 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398704:0:48385:0:(rw26.c:875:ll_write_end()) pos 835584, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398707:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008a149a64[3 00000000bdb38050 2 1 00000000746d8de4 0 0 204] 00000080:00200000:0.0:1787827241.398709:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000029bb6d2f 17ffffc0001001 3:0 ffff9a7886958bc8 204 lru 00000080:00200000:0.0:1787827241.398713:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006e52c514 204: 1< 0 - - > 2< 835584 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398715:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008a149a64 00000080:00200000:0.0:1787827241.398716:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 205. 00000080:00200000:0.0:1787827241.398717:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 205 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398734:0:48385:0:(rw26.c:875:ll_write_end()) pos 839680, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398737:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000210dbf49[3 00000000bdb38050 2 1 00000000746d8de4 0 0 205] 00000080:00200000:0.0:1787827241.398739:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000028cb3732 17ffffc0001021 2:0 ffff9a7890a37658 205 lru 00000080:00200000:0.0:1787827241.398742:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000344b7b71 205: 1< 0 - - > 2< 839680 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398743:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000210dbf49 00000080:00200000:0.0:1787827241.398744:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 206. 00000080:00200000:0.0:1787827241.398746:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 206 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398755:0:48385:0:(rw26.c:875:ll_write_end()) pos 843776, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398758:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ed467749[3 00000000bdb38050 2 1 00000000746d8de4 0 0 206] 00000080:00200000:0.0:1787827241.398760:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007ec4c8ea 17ffffc0001001 3:0 ffff9a7890a379f8 206 lru 00000080:00200000:0.0:1787827241.398764:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f2d92db1 206: 1< 0 - - > 2< 843776 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398765:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ed467749 00000080:00200000:0.0:1787827241.398766:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 207. 00000080:00200000:0.0:1787827241.398767:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 207 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398775:0:48385:0:(rw26.c:875:ll_write_end()) pos 847872, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398778:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000027b3546c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 207] 00000080:00200000:0.0:1787827241.398779:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e3f99f77 17ffffc0001001 3:0 ffff9a7890a37910 207 lru 00000080:00200000:0.0:1787827241.398783:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e2e08dd6 207: 1< 0 - - > 2< 847872 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398785:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000027b3546c 00000080:00200000:0.0:1787827241.398785:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 208. 00000080:00200000:0.0:1787827241.398787:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 208 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398795:0:48385:0:(rw26.c:875:ll_write_end()) pos 851968, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398798:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ae344a38[3 00000000bdb38050 2 1 00000000746d8de4 0 0 208] 00000080:00200000:0.0:1787827241.398800:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000058dd88db 17ffffc0001001 3:0 ffff9a7890a37d98 208 lru 00000080:00200000:0.0:1787827241.398803:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000001aa5443 208: 1< 0 - - > 2< 851968 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398804:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ae344a38 00000080:00200000:0.0:1787827241.398805:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 209. 00000080:00200000:0.0:1787827241.398806:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 209 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398814:0:48385:0:(rw26.c:875:ll_write_end()) pos 856064, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398817:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000755619af[3 00000000bdb38050 2 1 00000000746d8de4 0 0 209] 00000080:00200000:0.0:1787827241.398819:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000727e55d6 17ffffc0001001 3:0 ffff9a7890a373a0 209 lru 00000080:00200000:0.0:1787827241.398822:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000017f5167f 209: 1< 0 - - > 2< 856064 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398823:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000755619af 00000080:00200000:0.0:1787827241.398824:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 210. 00000080:00200000:0.0:1787827241.398825:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 210 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398855:0:48385:0:(rw26.c:875:ll_write_end()) pos 860160, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398861:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c747f3d2[3 00000000bdb38050 2 1 00000000746d8de4 0 0 210] 00000080:00200000:0.0:1787827241.398864:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000929415bd 17ffffc0001001 3:0 ffff9a7890a370e8 210 lru 00000080:00200000:0.0:1787827241.398870:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000efc2ce20 210: 1< 0 - - > 2< 860160 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398873:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c747f3d2 00000080:00200000:0.0:1787827241.398874:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 211. 00000080:00200000:0.0:1787827241.398876:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 211 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398893:0:48385:0:(rw26.c:875:ll_write_end()) pos 864256, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398898:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000037e7713d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 211] 00000080:00200000:0.0:1787827241.398900:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007ec37b70 17ffffc0001001 3:0 ffff9a7890a372b8 211 lru 00000080:00200000:0.0:1787827241.398906:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000047b2a66b 211: 1< 0 - - > 2< 864256 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398909:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000037e7713d 00000080:00200000:0.0:1787827241.398910:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 212. 00000080:00200000:0.0:1787827241.398911:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 212 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398923:0:48385:0:(rw26.c:875:ll_write_end()) pos 868352, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398926:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000efec2c78[3 00000000bdb38050 2 1 00000000746d8de4 0 0 212] 00000080:00200000:0.0:1787827241.398928:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a2b0028d 17ffffc0001001 3:0 ffff9a7890a371d0 212 lru 00000080:00200000:0.0:1787827241.398932:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d9e457cd 212: 1< 0 - - > 2< 868352 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398933:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000efec2c78 00000080:00200000:0.0:1787827241.398934:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 213. 00000080:00200000:0.0:1787827241.398935:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 213 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398943:0:48385:0:(rw26.c:875:ll_write_end()) pos 872448, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398946:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000513f0afa[3 00000000bdb38050 2 1 00000000746d8de4 0 0 213] 00000080:00200000:0.0:1787827241.398947:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000004e9bad0 17ffffc0001001 3:0 ffff9a7890a37740 213 lru 00000080:00200000:0.0:1787827241.398951:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006843a85d 213: 1< 0 - - > 2< 872448 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398952:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000513f0afa 00000080:00200000:0.0:1787827241.398953:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 214. 00000080:00200000:0.0:1787827241.398954:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 214 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398963:0:48385:0:(rw26.c:875:ll_write_end()) pos 876544, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398966:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000056078915[3 00000000bdb38050 2 1 00000000746d8de4 0 0 214] 00000080:00200000:0.0:1787827241.398968:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002d60dd07 17ffffc0001001 3:0 ffff9a7890a37e80 214 lru 00000080:00200000:0.0:1787827241.398972:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ec7595a0 214: 1< 0 - - > 2< 876544 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398973:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000056078915 00000080:00200000:0.0:1787827241.398974:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 215. 00000080:00200000:0.0:1787827241.398975:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 215 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.398983:0:48385:0:(rw26.c:875:ll_write_end()) pos 880640, len 4096, copied 4096 00000080:00200000:0.0:1787827241.398986:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a8f7f264[3 00000000bdb38050 2 1 00000000746d8de4 0 0 215] 00000080:00200000:0.0:1787827241.398988:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000034d740a 17ffffc0001001 3:0 ffff9a7890a37488 215 lru 00000080:00200000:0.0:1787827241.398991:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001f34d003 215: 1< 0 - - > 2< 880640 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.398993:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a8f7f264 00000080:00200000:0.0:1787827241.398993:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 216. 00000080:00200000:0.0:1787827241.398994:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 216 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399002:0:48385:0:(rw26.c:875:ll_write_end()) pos 884736, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399005:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fadee31f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 216] 00000080:00200000:0.0:1787827241.399006:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000002db4af0 17ffffc0001001 3:0 ffff9a7890a37bc8 216 lru 00000080:00200000:0.0:1787827241.399010:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fb769f4f 216: 1< 0 - - > 2< 884736 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399011:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fadee31f 00000080:00200000:0.0:1787827241.399012:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 217. 00000080:00200000:0.0:1787827241.399013:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 217 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399022:0:48385:0:(rw26.c:875:ll_write_end()) pos 888832, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399025:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000010b5c0b4[3 00000000bdb38050 2 1 00000000746d8de4 0 0 217] 00000080:00200000:0.0:1787827241.399027:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007f396d4b 17ffffc0001001 3:0 ffff9a7890a37570 217 lru 00000080:00200000:0.0:1787827241.399031:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d38cf57f 217: 1< 0 - - > 2< 888832 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399032:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000010b5c0b4 00000080:00200000:0.0:1787827241.399033:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 218. 00000080:00200000:0.0:1787827241.399034:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 218 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399042:0:48385:0:(rw26.c:875:ll_write_end()) pos 892928, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399044:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000022e4beda[3 00000000bdb38050 2 1 00000000746d8de4 0 0 218] 00000080:00200000:0.0:1787827241.399046:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000021177da3 17ffffc0001001 3:0 ffff9a7890a37828 218 lru 00000080:00200000:0.0:1787827241.399050:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000087ef44d4 218: 1< 0 - - > 2< 892928 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399051:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000022e4beda 00000080:00200000:0.0:1787827241.399052:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 219. 00000080:00200000:0.0:1787827241.399053:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 219 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399061:0:48385:0:(rw26.c:875:ll_write_end()) pos 897024, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399064:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000081c118c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 219] 00000080:00200000:0.0:1787827241.399066:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000394e1774 17ffffc0001001 3:0 ffff9a7890a37000 219 lru 00000080:00200000:0.0:1787827241.399070:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000c48d4fa 219: 1< 0 - - > 2< 897024 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399072:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000081c118c 00000080:00200000:0.0:1787827241.399073:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 220. 00000080:00200000:0.0:1787827241.399074:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 220 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399087:0:48385:0:(rw26.c:875:ll_write_end()) pos 901120, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399092:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000034521bd[3 00000000bdb38050 2 1 00000000746d8de4 0 0 220] 00000080:00200000:0.0:1787827241.399095:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000012409dca 17ffffc0001021 2:0 ffff9a7890a37cb0 220 lru 00000080:00200000:0.0:1787827241.399101:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000008ffa10d 220: 1< 0 - - > 2< 901120 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399104:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000034521bd 00000080:00200000:0.0:1787827241.399105:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 221. 00000080:00200000:0.0:1787827241.399107:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 221 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399122:0:48385:0:(rw26.c:875:ll_write_end()) pos 905216, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399127:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c1a446b0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 221] 00000080:00200000:0.0:1787827241.399130:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001f78f2c0 17ffffc0001001 3:0 ffff9a7890a37ae0 221 lru 00000080:00200000:0.0:1787827241.399136:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b43e6da8 221: 1< 0 - - > 2< 905216 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399137:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c1a446b0 00000080:00200000:0.0:1787827241.399138:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 222. 00000080:00200000:0.0:1787827241.399140:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 222 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399159:0:48385:0:(rw26.c:875:ll_write_end()) pos 909312, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399162:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002e05bc5e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 222] 00000080:00200000:0.0:1787827241.399164:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a3e80e83 17ffffc0001001 3:0 ffff9a78a66c20e8 222 lru 00000080:00200000:0.0:1787827241.399168:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005e51bc64 222: 1< 0 - - > 2< 909312 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399169:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002e05bc5e 00000080:00200000:0.0:1787827241.399170:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 223. 00000080:00200000:0.0:1787827241.399171:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 223 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399180:0:48385:0:(rw26.c:875:ll_write_end()) pos 913408, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399182:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c05d76aa[3 00000000bdb38050 2 1 00000000746d8de4 0 0 223] 00000080:00200000:0.0:1787827241.399184:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000802b2877 17ffffc0001001 3:0 ffff9a78a66c22b8 223 lru 00000080:00200000:0.0:1787827241.399188:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008349a0ed 223: 1< 0 - - > 2< 913408 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399189:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c05d76aa 00000080:00200000:0.0:1787827241.399190:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 224. 00000080:00200000:0.0:1787827241.399192:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 224 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399202:0:48385:0:(rw26.c:875:ll_write_end()) pos 917504, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399204:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000614a85f3[3 00000000bdb38050 2 1 00000000746d8de4 0 0 224] 00000080:00200000:0.0:1787827241.399206:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ccf7a45c 17ffffc0001001 3:0 ffff9a78a66c21d0 224 lru 00000080:00200000:0.0:1787827241.399210:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000018498062 224: 1< 0 - - > 2< 917504 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399212:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000614a85f3 00000080:00200000:0.0:1787827241.399212:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 225. 00000080:00200000:0.0:1787827241.399213:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 225 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399222:0:48385:0:(rw26.c:875:ll_write_end()) pos 921600, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399224:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e0132efb[3 00000000bdb38050 2 1 00000000746d8de4 0 0 225] 00000080:00200000:0.0:1787827241.399226:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000056fac3cf 17ffffc0001001 3:0 ffff9a78a66c2740 225 lru 00000080:00200000:0.0:1787827241.399229:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000059e6ed6a 225: 1< 0 - - > 2< 921600 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399231:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e0132efb 00000080:00200000:0.0:1787827241.399232:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 226. 00000080:00200000:0.0:1787827241.399233:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 226 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399241:0:48385:0:(rw26.c:875:ll_write_end()) pos 925696, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399244:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000af134c43[3 00000000bdb38050 2 1 00000000746d8de4 0 0 226] 00000080:00200000:0.0:1787827241.399246:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e2c623d5 17ffffc0001001 3:0 ffff9a78a66c2e80 226 lru 00000080:00200000:0.0:1787827241.399250:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007baf76b7 226: 1< 0 - - > 2< 925696 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399251:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000af134c43 00000080:00200000:0.0:1787827241.399252:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 227. 00000080:00200000:0.0:1787827241.399253:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 227 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399261:0:48385:0:(rw26.c:875:ll_write_end()) pos 929792, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399264:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bf0d3392[3 00000000bdb38050 2 1 00000000746d8de4 0 0 227] 00000080:00200000:0.0:1787827241.399265:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000051ea5b2f 17ffffc0001001 3:0 ffff9a78a66c2488 227 lru 00000080:00200000:0.0:1787827241.399269:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a28259ae 227: 1< 0 - - > 2< 929792 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399270:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bf0d3392 00000080:00200000:0.0:1787827241.399271:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 228. 00000080:00200000:0.0:1787827241.399272:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 228 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399281:0:48385:0:(rw26.c:875:ll_write_end()) pos 933888, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399283:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d4d905f2[3 00000000bdb38050 2 1 00000000746d8de4 0 0 228] 00000080:00200000:0.0:1787827241.399285:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000386e133c 17ffffc0001001 3:0 ffff9a78a66c2bc8 228 lru 00000080:00200000:0.0:1787827241.399288:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d63abd38 228: 1< 0 - - > 2< 933888 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399291:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d4d905f2 00000080:00200000:0.0:1787827241.399292:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 229. 00000080:00200000:0.0:1787827241.399293:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 229 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399301:0:48385:0:(rw26.c:875:ll_write_end()) pos 937984, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399303:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001b0b5afa[3 00000000bdb38050 2 1 00000000746d8de4 0 0 229] 00000080:00200000:0.0:1787827241.399305:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009d31aa99 17ffffc0001001 3:0 ffff9a78a66c2570 229 lru 00000080:00200000:0.0:1787827241.399309:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e6e14bc8 229: 1< 0 - - > 2< 937984 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399310:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001b0b5afa 00000080:00200000:0.0:1787827241.399310:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 230. 00000080:00200000:0.0:1787827241.399312:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 230 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399320:0:48385:0:(rw26.c:875:ll_write_end()) pos 942080, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399323:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007d6bc6a8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 230] 00000080:00200000:0.0:1787827241.399324:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ee98b02b 17ffffc0001001 3:0 ffff9a78a66c2828 230 lru 00000080:00200000:0.0:1787827241.399328:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009720d607 230: 1< 0 - - > 2< 942080 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399329:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007d6bc6a8 00000080:00200000:0.0:1787827241.399329:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 231. 00000080:00200000:0.0:1787827241.399331:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 231 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399340:0:48385:0:(rw26.c:875:ll_write_end()) pos 946176, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399342:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009b81a065[3 00000000bdb38050 2 1 00000000746d8de4 0 0 231] 00000080:00200000:0.0:1787827241.399344:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000032956f1c 17ffffc0001001 3:0 ffff9a78a66c2000 231 lru 00000080:00200000:0.0:1787827241.399347:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000023d4c1fb 231: 1< 0 - - > 2< 946176 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399349:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009b81a065 00000080:00200000:0.0:1787827241.399350:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 232. 00000080:00200000:0.0:1787827241.399351:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 232 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399359:0:48385:0:(rw26.c:875:ll_write_end()) pos 950272, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399361:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000eaf89116[3 00000000bdb38050 2 1 00000000746d8de4 0 0 232] 00000080:00200000:0.0:1787827241.399363:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c425b1fe 17ffffc0001001 3:0 ffff9a78a66c2cb0 232 lru 00000080:00200000:0.0:1787827241.399367:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c2c51e51 232: 1< 0 - - > 2< 950272 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399368:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000eaf89116 00000080:00200000:0.0:1787827241.399368:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 233. 00000080:00200000:0.0:1787827241.399369:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 233 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399378:0:48385:0:(rw26.c:875:ll_write_end()) pos 954368, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399381:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000500b82a3[3 00000000bdb38050 2 1 00000000746d8de4 0 0 233] 00000080:00200000:0.0:1787827241.399383:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000efa3e249 17ffffc0001001 3:0 ffff9a78a66c2ae0 233 lru 00000080:00200000:0.0:1787827241.399386:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005c20e67b 233: 1< 0 - - > 2< 954368 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399387:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000500b82a3 00000080:00200000:0.0:1787827241.399388:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 234. 00000080:00200000:0.0:1787827241.399389:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 234 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399397:0:48385:0:(rw26.c:875:ll_write_end()) pos 958464, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399399:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b42dc4a6[3 00000000bdb38050 2 1 00000000746d8de4 0 0 234] 00000080:00200000:0.0:1787827241.399401:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002d5f7c7b 17ffffc0001001 3:0 ffff9a78a66c2658 234 lru 00000080:00200000:0.0:1787827241.399404:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000aa55a95b 234: 1< 0 - - > 2< 958464 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399406:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b42dc4a6 00000080:00200000:0.0:1787827241.399406:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 235. 00000080:00200000:0.0:1787827241.399407:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 235 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399417:0:48385:0:(rw26.c:875:ll_write_end()) pos 962560, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399420:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005eb013d8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 235] 00000080:00200000:0.0:1787827241.399422:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e32192d4 17ffffc0001021 2:0 ffff9a78a66c29f8 235 lru 00000080:00200000:0.0:1787827241.399426:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009e51bef4 235: 1< 0 - - > 2< 962560 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399427:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005eb013d8 00000080:00200000:0.0:1787827241.399428:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 236. 00000080:00200000:0.0:1787827241.399429:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 236 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399437:0:48385:0:(rw26.c:875:ll_write_end()) pos 966656, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399440:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001b673138[3 00000000bdb38050 2 1 00000000746d8de4 0 0 236] 00000080:00200000:0.0:1787827241.399442:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002ee59700 17ffffc0001001 3:0 ffff9a78a66c2910 236 lru 00000080:00200000:0.0:1787827241.399445:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000056cde5da 236: 1< 0 - - > 2< 966656 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399447:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001b673138 00000080:00200000:0.0:1787827241.399448:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 237. 00000080:00200000:0.0:1787827241.399449:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 237 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399457:0:48385:0:(rw26.c:875:ll_write_end()) pos 970752, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399459:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000029b214da[3 00000000bdb38050 2 1 00000000746d8de4 0 0 237] 00000080:00200000:0.0:1787827241.399461:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f6e9af7a 17ffffc0001001 3:0 ffff9a78a66c2d98 237 lru 00000080:00200000:0.0:1787827241.399465:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000042506977 237: 1< 0 - - > 2< 970752 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399467:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000029b214da 00000080:00200000:0.0:1787827241.399468:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 238. 00000080:00200000:0.0:1787827241.399469:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 238 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399480:0:48385:0:(rw26.c:875:ll_write_end()) pos 974848, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399482:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000606ccd0b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 238] 00000080:00200000:0.0:1787827241.399484:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000edd56aa5 17ffffc0001001 3:0 ffff9a78a66c23a0 238 lru 00000080:00200000:0.0:1787827241.399488:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000062f3b53f 238: 1< 0 - - > 2< 974848 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399490:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000606ccd0b 00000080:00200000:0.0:1787827241.399490:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 239. 00000080:00200000:0.0:1787827241.399491:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 239 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399520:0:48385:0:(rw26.c:875:ll_write_end()) pos 978944, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399523:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002589044a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 239] 00000080:00200000:0.0:1787827241.399525:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001e3e63ad 17ffffc0001001 3:0 ffff9a7888076488 239 lru 00000080:00200000:0.0:1787827241.399528:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000aca4ee40 239: 1< 0 - - > 2< 978944 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399529:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002589044a 00000080:00200000:0.0:1787827241.399530:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 240. 00000080:00200000:0.0:1787827241.399532:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 240 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399543:0:48385:0:(rw26.c:875:ll_write_end()) pos 983040, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399545:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ea5719b9[3 00000000bdb38050 2 1 00000000746d8de4 0 0 240] 00000080:00200000:0.0:1787827241.399547:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b102dbfc 17ffffc0001001 3:0 ffff9a7888076bc8 240 lru 00000080:00200000:0.0:1787827241.399551:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000039eea8d2 240: 1< 0 - - > 2< 983040 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399553:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ea5719b9 00000080:00200000:0.0:1787827241.399554:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 241. 00000080:00200000:0.0:1787827241.399555:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 241 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399563:0:48385:0:(rw26.c:875:ll_write_end()) pos 987136, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399565:0:48385:0:(rw26.c:933:ll_write_end()) page@000000003fba9d0b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 241] 00000080:00200000:0.0:1787827241.399567:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a917cb6c 17ffffc0001001 3:0 ffff9a7888076570 241 lru 00000080:00200000:0.0:1787827241.399571:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000e8fecdd 241: 1< 0 - - > 2< 987136 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399572:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000003fba9d0b 00000080:00200000:0.0:1787827241.399573:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 242. 00000080:00200000:0.0:1787827241.399574:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 242 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399584:0:48385:0:(rw26.c:875:ll_write_end()) pos 991232, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399587:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000068705a4f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 242] 00000080:00200000:0.0:1787827241.399589:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000720a9d89 17ffffc0001001 3:0 ffff9a7888076828 242 lru 00000080:00200000:0.0:1787827241.399593:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000087761bd 242: 1< 0 - - > 2< 991232 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399594:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000068705a4f 00000080:00200000:0.0:1787827241.399595:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 243. 00000080:00200000:0.0:1787827241.399596:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 243 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399604:0:48385:0:(rw26.c:875:ll_write_end()) pos 995328, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399606:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a75f6def[3 00000000bdb38050 2 1 00000000746d8de4 0 0 243] 00000080:00200000:0.0:1787827241.399608:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000bb2e36e4 17ffffc0001001 3:0 ffff9a7888076000 243 lru 00000080:00200000:0.0:1787827241.399611:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c5546a58 243: 1< 0 - - > 2< 995328 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399613:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a75f6def 00000080:00200000:0.0:1787827241.399613:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 244. 00000080:00200000:0.0:1787827241.399614:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 244 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399623:0:48385:0:(rw26.c:875:ll_write_end()) pos 999424, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399625:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009e1545c0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 244] 00000080:00200000:0.0:1787827241.399627:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f98c4759 17ffffc0001001 3:0 ffff9a7888076cb0 244 lru 00000080:00200000:0.0:1787827241.399630:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000030f6678d 244: 1< 0 - - > 2< 999424 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399632:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009e1545c0 00000080:00200000:0.0:1787827241.399632:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 245. 00000080:00200000:0.0:1787827241.399633:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 245 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399643:0:48385:0:(rw26.c:875:ll_write_end()) pos 1003520, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399645:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000032a48a2f[3 00000000bdb38050 2 1 00000000746d8de4 0 0 245] 00000080:00200000:0.0:1787827241.399647:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004f1a35ec 17ffffc0001001 3:0 ffff9a7888076ae0 245 lru 00000080:00200000:0.0:1787827241.399650:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000270c74a0 245: 1< 0 - - > 2< 1003520 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399652:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000032a48a2f 00000080:00200000:0.0:1787827241.399652:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 246. 00000080:00200000:0.0:1787827241.399653:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 246 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399662:0:48385:0:(rw26.c:875:ll_write_end()) pos 1007616, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399664:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d53bfcc8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 246] 00000080:00200000:0.0:1787827241.399666:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000051bd0560 17ffffc0001001 3:0 ffff9a7888076658 246 lru 00000080:00200000:0.0:1787827241.399669:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001ce910c7 246: 1< 0 - - > 2< 1007616 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399671:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d53bfcc8 00000080:00200000:0.0:1787827241.399672:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 247. 00000080:00200000:0.0:1787827241.399674:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 247 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399682:0:48385:0:(rw26.c:875:ll_write_end()) pos 1011712, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399685:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000057ddff6[3 00000000bdb38050 2 1 00000000746d8de4 0 0 247] 00000080:00200000:0.0:1787827241.399687:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000045b0a237 17ffffc0001001 3:0 ffff9a78880769f8 247 lru 00000080:00200000:0.0:1787827241.399691:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000915e3b32 247: 1< 0 - - > 2< 1011712 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399692:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000057ddff6 00000080:00200000:0.0:1787827241.399693:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 248. 00000080:00200000:0.0:1787827241.399694:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 248 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399701:0:48385:0:(rw26.c:875:ll_write_end()) pos 1015808, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399704:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000081ca67c5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 248] 00000080:00200000:0.0:1787827241.399706:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000046b857 17ffffc0001001 3:0 ffff9a7888076910 248 lru 00000080:00200000:0.0:1787827241.399709:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000052d46914 248: 1< 0 - - > 2< 1015808 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399710:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000081ca67c5 00000080:00200000:0.0:1787827241.399711:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 249. 00000080:00200000:0.0:1787827241.399712:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 249 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399720:0:48385:0:(rw26.c:875:ll_write_end()) pos 1019904, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399722:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000782a8c67[3 00000000bdb38050 2 1 00000000746d8de4 0 0 249] 00000080:00200000:0.0:1787827241.399725:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002ac753b2 17ffffc0001001 3:0 ffff9a7888076d98 249 lru 00000080:00200000:0.0:1787827241.399731:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d7b5087d 249: 1< 0 - - > 2< 1019904 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399733:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000782a8c67 00000080:00200000:0.0:1787827241.399734:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 250. 00000080:00200000:0.0:1787827241.399736:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 250 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399753:0:48385:0:(rw26.c:875:ll_write_end()) pos 1024000, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399758:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001b2e06a1[3 00000000bdb38050 2 1 00000000746d8de4 0 0 250] 00000080:00200000:0.0:1787827241.399761:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000071b1e3a1 17ffffc0001021 2:0 ffff9a78880763a0 250 lru 00000080:00200000:0.0:1787827241.399767:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000056c9e5bb 250: 1< 0 - - > 2< 1024000 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399770:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001b2e06a1 00000080:00200000:0.0:1787827241.399771:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 251. 00000080:00200000:0.0:1787827241.399773:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 251 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399788:0:48385:0:(rw26.c:875:ll_write_end()) pos 1028096, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399791:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000057fa4fd8[3 00000000bdb38050 2 1 00000000746d8de4 0 0 251] 00000080:00200000:0.0:1787827241.399793:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000752448dd 17ffffc0001001 3:0 ffff9a78880760e8 251 lru 00000080:00200000:0.0:1787827241.399797:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000619eb806 251: 1< 0 - - > 2< 1028096 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399799:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000057fa4fd8 00000080:00200000:0.0:1787827241.399800:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 252. 00000080:00200000:0.0:1787827241.399801:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 252 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399811:0:48385:0:(rw26.c:875:ll_write_end()) pos 1032192, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399813:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008208cc3c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 252] 00000080:00200000:0.0:1787827241.399815:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000049ae7bdf 17ffffc0001001 3:0 ffff9a78880762b8 252 lru 00000080:00200000:0.0:1787827241.399819:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c1d726a4 252: 1< 0 - - > 2< 1032192 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399821:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008208cc3c 00000080:00200000:0.0:1787827241.399822:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 253. 00000080:00200000:0.0:1787827241.399823:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 253 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399848:0:48385:0:(rw26.c:875:ll_write_end()) pos 1036288, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399854:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e36bbb8e[3 00000000bdb38050 2 1 00000000746d8de4 0 0 253] 00000080:00200000:0.0:1787827241.399857:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005a562051 17ffffc0001001 3:0 ffff9a78880761d0 253 lru 00000080:00200000:0.0:1787827241.399863:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ca64850a 253: 1< 0 - - > 2< 1036288 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399866:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e36bbb8e 00000080:00200000:0.0:1787827241.399868:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 254. 00000080:00200000:0.0:1787827241.399870:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 254 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399886:0:48385:0:(rw26.c:875:ll_write_end()) pos 1040384, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399891:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000055490eda[3 00000000bdb38050 2 1 00000000746d8de4 0 0 254] 00000080:00200000:0.0:1787827241.399895:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003015b41a 17ffffc0001001 3:0 ffff9a7888076740 254 lru 00000080:00200000:0.0:1787827241.399900:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003dee75db 254: 1< 0 - - > 2< 1040384 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399903:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000055490eda 00000080:00200000:0.0:1787827241.399904:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 255. 00000080:00200000:0.0:1787827241.399906:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 255 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399921:0:48385:0:(rw26.c:875:ll_write_end()) pos 1044480, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399925:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e55c1516[3 00000000bdb38050 2 1 00000000746d8de4 0 0 255] 00000080:00200000:0.0:1787827241.399927:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a3175f10 17ffffc0001001 3:0 ffff9a7888076e80 255 lru 00000080:00200000:0.0:1787827241.399931:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a129c2f7 255: 1< 0 - - > 2< 1044480 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399932:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e55c1516 00000080:00200000:0.0:1787827241.399933:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 256. 00000080:00200000:0.0:1787827241.399935:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 256 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399953:0:48385:0:(rw26.c:875:ll_write_end()) pos 1048576, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399956:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000075b47923[3 00000000bdb38050 2 1 00000000746d8de4 0 0 256] 00000080:00200000:0.0:1787827241.399957:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000099d33471 17ffffc0001001 3:0 ffff9a788626ed98 256 lru 00000080:00200000:0.0:1787827241.399961:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000eb798afa 256: 1< 0 - - > 2< 1048576 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399963:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000075b47923 00000080:00200000:0.0:1787827241.399964:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 257. 00000080:00200000:0.0:1787827241.399965:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 257 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399974:0:48385:0:(rw26.c:875:ll_write_end()) pos 1052672, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399977:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000004ec9c20[3 00000000bdb38050 2 1 00000000746d8de4 0 0 257] 00000080:00200000:0.0:1787827241.399979:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002695e4b5 17ffffc0001001 3:0 ffff9a788626e3a0 257 lru 00000080:00200000:0.0:1787827241.399983:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000095ed7a8e 257: 1< 0 - - > 2< 1052672 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.399984:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000004ec9c20 00000080:00200000:0.0:1787827241.399985:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 258. 00000080:00200000:0.0:1787827241.399986:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 258 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.399995:0:48385:0:(rw26.c:875:ll_write_end()) pos 1056768, len 4096, copied 4096 00000080:00200000:0.0:1787827241.399998:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004ae3ac85[3 00000000bdb38050 2 1 00000000746d8de4 0 0 258] 00000080:00200000:0.0:1787827241.399999:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d614da87 17ffffc0001001 3:0 ffff9a788626e0e8 258 lru 00000080:00200000:0.0:1787827241.400003:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000faad386f 258: 1< 0 - - > 2< 1056768 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400004:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004ae3ac85 00000080:00200000:0.0:1787827241.400005:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 259. 00000080:00200000:0.0:1787827241.400006:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 259 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400016:0:48385:0:(rw26.c:875:ll_write_end()) pos 1060864, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400018:0:48385:0:(rw26.c:933:ll_write_end()) page@000000001ce0486d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 259] 00000080:00200000:0.0:1787827241.400020:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ffa2c79d 17ffffc0001001 3:0 ffff9a788626e2b8 259 lru 00000080:00200000:0.0:1787827241.400024:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d9aadf30 259: 1< 0 - - > 2< 1060864 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400025:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000001ce0486d 00000080:00200000:0.0:1787827241.400027:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 260. 00000080:00200000:0.0:1787827241.400028:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 260 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400036:0:48385:0:(rw26.c:875:ll_write_end()) pos 1064960, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400039:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d0b2b411[3 00000000bdb38050 2 1 00000000746d8de4 0 0 260] 00000080:00200000:0.0:1787827241.400041:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009719cde9 17ffffc0001001 3:0 ffff9a788626e1d0 260 lru 00000080:00200000:0.0:1787827241.400045:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007d0de4ce 260: 1< 0 - - > 2< 1064960 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400046:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d0b2b411 00000080:00200000:0.0:1787827241.400047:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 261. 00000080:00200000:0.0:1787827241.400048:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 261 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400057:0:48385:0:(rw26.c:875:ll_write_end()) pos 1069056, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400059:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dbcd81ea[3 00000000bdb38050 2 1 00000000746d8de4 0 0 261] 00000080:00200000:0.0:1787827241.400061:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008901090d 17ffffc0001001 3:0 ffff9a788626e740 261 lru 00000080:00200000:0.0:1787827241.400065:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000661448e8 261: 1< 0 - - > 2< 1069056 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400066:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dbcd81ea 00000080:00200000:0.0:1787827241.400067:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 262. 00000080:00200000:0.0:1787827241.400068:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 262 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400076:0:48385:0:(rw26.c:875:ll_write_end()) pos 1073152, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400079:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009519faa7[3 00000000bdb38050 2 1 00000000746d8de4 0 0 262] 00000080:00200000:0.0:1787827241.400082:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000052be8063 17ffffc0001001 3:0 ffff9a788626ee80 262 lru 00000080:00200000:0.0:1787827241.400088:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007668b8c5 262: 1< 0 - - > 2< 1073152 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400090:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009519faa7 00000080:00200000:0.0:1787827241.400092:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 263. 00000080:00200000:0.0:1787827241.400094:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 263 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400109:0:48385:0:(rw26.c:875:ll_write_end()) pos 1077248, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400114:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b20a9d83[3 00000000bdb38050 2 1 00000000746d8de4 0 0 263] 00000080:00200000:0.0:1787827241.400117:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000966d7f88 17ffffc0001001 3:0 ffff9a788626e488 263 lru 00000080:00200000:0.0:1787827241.400124:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000067ce92f1 263: 1< 0 - - > 2< 1077248 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400127:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b20a9d83 00000080:00200000:0.0:1787827241.400128:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 264. 00000080:00200000:0.0:1787827241.400130:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 264 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400144:0:48385:0:(rw26.c:875:ll_write_end()) pos 1081344, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400147:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000011f12153[3 00000000bdb38050 2 1 00000000746d8de4 0 0 264] 00000080:00200000:0.0:1787827241.400149:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a64d00c9 17ffffc0001001 3:0 ffff9a788626ebc8 264 lru 00000080:00200000:0.0:1787827241.400153:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c9f1726d 264: 1< 0 - - > 2< 1081344 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400155:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000011f12153 00000080:00200000:0.0:1787827241.400156:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 265. 00000080:00200000:0.0:1787827241.400157:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 265 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400168:0:48385:0:(rw26.c:875:ll_write_end()) pos 1085440, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400171:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b56e8bdf[3 00000000bdb38050 2 1 00000000746d8de4 0 0 265] 00000080:00200000:0.0:1787827241.400173:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003c8027fe 17ffffc0001021 2:0 ffff9a788626e570 265 lru 00000080:00200000:0.0:1787827241.400176:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d930601b 265: 1< 0 - - > 2< 1085440 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400178:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b56e8bdf 00000080:00200000:0.0:1787827241.400179:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 266. 00000080:00200000:0.0:1787827241.400180:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 266 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400190:0:48385:0:(rw26.c:875:ll_write_end()) pos 1089536, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400192:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000903adb15[3 00000000bdb38050 2 1 00000000746d8de4 0 0 266] 00000080:00200000:0.0:1787827241.400194:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000086629071 17ffffc0001001 3:0 ffff9a788626e828 266 lru 00000080:00200000:0.0:1787827241.400198:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a738cd76 266: 1< 0 - - > 2< 1089536 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400199:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000903adb15 00000080:00200000:0.0:1787827241.400200:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 267. 00000080:00200000:0.0:1787827241.400202:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 267 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400210:0:48385:0:(rw26.c:875:ll_write_end()) pos 1093632, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400212:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bc3cf61d[3 00000000bdb38050 2 1 00000000746d8de4 0 0 267] 00000080:00200000:0.0:1787827241.400214:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009019719b 17ffffc0001001 3:0 ffff9a788626e000 267 lru 00000080:00200000:0.0:1787827241.400218:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000facf486 267: 1< 0 - - > 2< 1093632 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400220:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bc3cf61d 00000080:00200000:0.0:1787827241.400221:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 268. 00000080:00200000:0.0:1787827241.400222:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 268 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400230:0:48385:0:(rw26.c:875:ll_write_end()) pos 1097728, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400232:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c2cc3b78[3 00000000bdb38050 2 1 00000000746d8de4 0 0 268] 00000080:00200000:0.0:1787827241.400234:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000dd6f826c 17ffffc0001001 3:0 ffff9a788626ecb0 268 lru 00000080:00200000:0.0:1787827241.400237:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b2d6347f 268: 1< 0 - - > 2< 1097728 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400240:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c2cc3b78 00000080:00200000:0.0:1787827241.400241:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 269. 00000080:00200000:0.0:1787827241.400242:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 269 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400250:0:48385:0:(rw26.c:875:ll_write_end()) pos 1101824, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400253:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009faeec48[3 00000000bdb38050 2 1 00000000746d8de4 0 0 269] 00000080:00200000:0.0:1787827241.400255:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cadeb894 17ffffc0001001 3:0 ffff9a788626eae0 269 lru 00000080:00200000:0.0:1787827241.400259:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b7218b5f 269: 1< 0 - - > 2< 1101824 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400260:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009faeec48 00000080:00200000:0.0:1787827241.400261:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 270. 00000080:00200000:0.0:1787827241.400262:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 270 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400270:0:48385:0:(rw26.c:875:ll_write_end()) pos 1105920, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400273:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004363388a[3 00000000bdb38050 2 1 00000000746d8de4 0 0 270] 00000080:00200000:0.0:1787827241.400274:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000d010f4f 17ffffc0001001 3:0 ffff9a788626e658 270 lru 00000080:00200000:0.0:1787827241.400278:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000000839ec0 270: 1< 0 - - > 2< 1105920 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400279:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004363388a 00000080:00200000:0.0:1787827241.400280:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 271. 00000080:00200000:0.0:1787827241.400281:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 271 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400289:0:48385:0:(rw26.c:875:ll_write_end()) pos 1110016, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400292:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009c83020b[3 00000000bdb38050 2 1 00000000746d8de4 0 0 271] 00000080:00200000:0.0:1787827241.400294:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000080ebfffc 17ffffc0001001 3:0 ffff9a788626e9f8 271 lru 00000080:00200000:0.0:1787827241.400298:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000074e3305e 271: 1< 0 - - > 2< 1110016 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400300:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009c83020b 00000080:00200000:0.0:1787827241.400300:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 272. 00000080:00200000:0.0:1787827241.400301:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 272 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400309:0:48385:0:(rw26.c:875:ll_write_end()) pos 1114112, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400312:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d439d7a9[3 00000000bdb38050 2 1 00000000746d8de4 0 0 272] 00000080:00200000:0.0:1787827241.400313:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000027ec48b7 17ffffc0001001 3:0 ffff9a788626e910 272 lru 00000080:00200000:0.0:1787827241.400317:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000a7580cf 272: 1< 0 - - > 2< 1114112 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400318:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d439d7a9 00000080:00200000:0.0:1787827241.400319:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 273. 00000080:00200000:0.0:1787827241.400320:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 273 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400337:0:48385:0:(rw26.c:875:ll_write_end()) pos 1118208, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400340:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009ea3cbc0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 273] 00000080:00200000:0.0:1787827241.400342:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000dcec9f77 17ffffc0001001 3:0 ffff9a7885932ae0 273 lru 00000080:00200000:0.0:1787827241.400346:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e3bf9ce9 273: 1< 0 - - > 2< 1118208 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400347:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009ea3cbc0 00000080:00200000:0.0:1787827241.400348:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 274. 00000080:00200000:0.0:1787827241.400349:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 274 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400358:0:48385:0:(rw26.c:875:ll_write_end()) pos 1122304, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400360:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000064fab2a0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 274] 00000080:00200000:0.0:1787827241.400362:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000dda67efd 17ffffc0001001 3:0 ffff9a7885932658 274 lru 00000080:00200000:0.0:1787827241.400365:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009c49cded 274: 1< 0 - - > 2< 1122304 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400367:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000064fab2a0 00000080:00200000:0.0:1787827241.400367:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 275. 00000080:00200000:0.0:1787827241.400368:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 275 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400376:0:48385:0:(rw26.c:875:ll_write_end()) pos 1126400, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400379:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000a51bfa5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 275] 00000080:00200000:0.0:1787827241.400380:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000070710a42 17ffffc0001001 3:0 ffff9a78859329f8 275 lru 00000080:00200000:0.0:1787827241.400384:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000be725315 275: 1< 0 - - > 2< 1126400 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400385:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000a51bfa5 00000080:00200000:0.0:1787827241.400386:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 276. 00000080:00200000:0.0:1787827241.400387:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 276 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400395:0:48385:0:(rw26.c:875:ll_write_end()) pos 1130496, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400397:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000012865b21[3 00000000bdb38050 2 1 00000000746d8de4 0 0 276] 00000080:00200000:0.0:1787827241.400399:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e5928faf 17ffffc0001001 3:0 ffff9a7885932910 276 lru 00000080:00200000:0.0:1787827241.400402:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000017315f70 276: 1< 0 - - > 2< 1130496 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400403:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000012865b21 00000080:00200000:0.0:1787827241.400404:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 277. 00000080:00200000:0.0:1787827241.400405:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 277 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400413:0:48385:0:(rw26.c:875:ll_write_end()) pos 1134592, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400416:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000077e76f39[3 00000000bdb38050 2 1 00000000746d8de4 0 0 277] 00000080:00200000:0.0:1787827241.400418:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f36f216b 17ffffc0001001 3:0 ffff9a7885932d98 277 lru 00000080:00200000:0.0:1787827241.400422:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a3c029a1 277: 1< 0 - - > 2< 1134592 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400424:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000077e76f39 00000080:00200000:0.0:1787827241.400425:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 278. 00000080:00200000:0.0:1787827241.400426:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 278 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400435:0:48385:0:(rw26.c:875:ll_write_end()) pos 1138688, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400437:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000003b3c4fb[3 00000000bdb38050 2 1 00000000746d8de4 0 0 278] 00000080:00200000:0.0:1787827241.400439:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c01fde9a 17ffffc0001001 3:0 ffff9a78859323a0 278 lru 00000080:00200000:0.0:1787827241.400442:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008f2a9f7f 278: 1< 0 - - > 2< 1138688 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400444:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000003b3c4fb 00000080:00200000:0.0:1787827241.400444:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 279. 00000080:00200000:0.0:1787827241.400446:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 279 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400454:0:48385:0:(rw26.c:875:ll_write_end()) pos 1142784, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400457:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000069b458ce[3 00000000bdb38050 2 1 00000000746d8de4 0 0 279] 00000080:00200000:0.0:1787827241.400458:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000042381746 17ffffc0001001 3:0 ffff9a78859320e8 279 lru 00000080:00200000:0.0:1787827241.400462:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005c170071 279: 1< 0 - - > 2< 1142784 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400463:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000069b458ce 00000080:00200000:0.0:1787827241.400464:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 280. 00000080:00200000:0.0:1787827241.400465:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 280 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400476:0:48385:0:(rw26.c:875:ll_write_end()) pos 1146880, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400479:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000010beaa5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 280] 00000080:00200000:0.0:1787827241.400481:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000046737b76 17ffffc0001021 2:0 ffff9a78859322b8 280 lru 00000080:00200000:0.0:1787827241.400484:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000b1085ec2 280: 1< 0 - - > 2< 1146880 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400485:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000010beaa5 00000080:00200000:0.0:1787827241.400486:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 281. 00000080:00200000:0.0:1787827241.400487:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 281 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400495:0:48385:0:(rw26.c:875:ll_write_end()) pos 1150976, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400498:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cccdbc30[3 00000000bdb38050 2 1 00000000746d8de4 0 0 281] 00000080:00200000:0.0:1787827241.400500:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000095b2b3fe 17ffffc0001001 3:0 ffff9a78859321d0 281 lru 00000080:00200000:0.0:1787827241.400516:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000051d8dd87 281: 1< 0 - - > 2< 1150976 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400518:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cccdbc30 00000080:00200000:0.0:1787827241.400519:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 282. 00000080:00200000:0.0:1787827241.400521:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 282 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400533:0:48385:0:(rw26.c:875:ll_write_end()) pos 1155072, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400536:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000068b39ed1[3 00000000bdb38050 2 1 00000000746d8de4 0 0 282] 00000080:00200000:0.0:1787827241.400538:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000d3f1d08 17ffffc0001001 3:0 ffff9a7885932740 282 lru 00000080:00200000:0.0:1787827241.400542:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000021b83bb4 282: 1< 0 - - > 2< 1155072 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400543:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000068b39ed1 00000080:00200000:0.0:1787827241.400544:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 283. 00000080:00200000:0.0:1787827241.400545:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 283 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400554:0:48385:0:(rw26.c:875:ll_write_end()) pos 1159168, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400556:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000095dda149[3 00000000bdb38050 2 1 00000000746d8de4 0 0 283] 00000080:00200000:0.0:1787827241.400558:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000001e00d41 17ffffc0001001 3:0 ffff9a7885932e80 283 lru 00000080:00200000:0.0:1787827241.400561:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a2eccd52 283: 1< 0 - - > 2< 1159168 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400563:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000095dda149 00000080:00200000:0.0:1787827241.400563:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 284. 00000080:00200000:0.0:1787827241.400565:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 284 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400573:0:48385:0:(rw26.c:875:ll_write_end()) pos 1163264, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400575:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000d267fd5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 284] 00000080:00200000:0.0:1787827241.400577:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003f191ac4 17ffffc0001001 3:0 ffff9a7885932488 284 lru 00000080:00200000:0.0:1787827241.400581:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001dd26011 284: 1< 0 - - > 2< 1163264 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400582:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000d267fd5 00000080:00200000:0.0:1787827241.400583:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 285. 00000080:00200000:0.0:1787827241.400585:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 285 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400593:0:48385:0:(rw26.c:875:ll_write_end()) pos 1167360, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400595:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000435e7211[3 00000000bdb38050 2 1 00000000746d8de4 0 0 285] 00000080:00200000:0.0:1787827241.400597:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000e649ccee 17ffffc0001001 3:0 ffff9a7885932bc8 285 lru 00000080:00200000:0.0:1787827241.400600:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a81a2db8 285: 1< 0 - - > 2< 1167360 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400601:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000435e7211 00000080:00200000:0.0:1787827241.400602:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 286. 00000080:00200000:0.0:1787827241.400603:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 286 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400611:0:48385:0:(rw26.c:875:ll_write_end()) pos 1171456, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400614:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000035df9072[3 00000000bdb38050 2 1 00000000746d8de4 0 0 286] 00000080:00200000:0.0:1787827241.400616:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007d985cc3 17ffffc0001001 3:0 ffff9a7885932570 286 lru 00000080:00200000:0.0:1787827241.400619:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005da93c3c 286: 1< 0 - - > 2< 1171456 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400621:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000035df9072 00000080:00200000:0.0:1787827241.400621:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 287. 00000080:00200000:0.0:1787827241.400622:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 287 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400632:0:48385:0:(rw26.c:875:ll_write_end()) pos 1175552, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400634:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005c47b12c[3 00000000bdb38050 2 1 00000000746d8de4 0 0 287] 00000080:00200000:0.0:1787827241.400636:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fde3a6b3 17ffffc0001001 3:0 ffff9a7885932828 287 lru 00000080:00200000:0.0:1787827241.400640:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006121b671 287: 1< 0 - - > 2< 1175552 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400641:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005c47b12c 00000080:00200000:0.0:1787827241.400641:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 288. 00000080:00200000:0.0:1787827241.400643:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 288 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400650:0:48385:0:(rw26.c:875:ll_write_end()) pos 1179648, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400653:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000fe489ba0[3 00000000bdb38050 2 1 00000000746d8de4 0 0 288] 00000080:00200000:0.0:1787827241.400654:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c8a5b7b5 17ffffc0001001 3:0 ffff9a7885932000 288 lru 00000080:00200000:0.0:1787827241.400658:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007010b323 288: 1< 0 - - > 2< 1179648 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400659:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000fe489ba0 00000080:00200000:0.0:1787827241.400659:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 289. 00000080:00200000:0.0:1787827241.400661:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 289 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400668:0:48385:0:(rw26.c:875:ll_write_end()) pos 1183744, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400671:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000dd7d4601[3 00000000bdb38050 2 1 00000000746d8de4 0 0 289] 00000080:00200000:0.0:1787827241.400672:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000239950c9 17ffffc0001001 3:0 ffff9a7885932cb0 289 lru 00000080:00200000:0.0:1787827241.400676:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d9a3a7a3 289: 1< 0 - - > 2< 1183744 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400677:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000dd7d4601 00000080:00200000:0.0:1787827241.400678:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 290. 00000080:00200000:0.0:1787827241.400679:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 290 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400693:0:48385:0:(rw26.c:875:ll_write_end()) pos 1187840, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400696:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000090045eee[3 00000000bdb38050 2 1 00000000746d8de4 0 0 290] 00000080:00200000:0.0:1787827241.400698:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d6eff288 17ffffc0001001 3:0 ffff9a788a7e5910 290 lru 00000080:00200000:0.0:1787827241.400702:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000078a7cff1 290: 1< 0 - - > 2< 1187840 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400705:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000090045eee 00000080:00200000:0.0:1787827241.400705:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 291. 00000080:00200000:0.0:1787827241.400707:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 291 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400715:0:48385:0:(rw26.c:875:ll_write_end()) pos 1191936, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400718:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a886e6ef[3 00000000bdb38050 2 1 00000000746d8de4 0 0 291] 00000080:00200000:0.0:1787827241.400720:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000108ee7ba 17ffffc0001001 3:0 ffff9a788a7e5d98 291 lru 00000080:00200000:0.0:1787827241.400726:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000eea1c10a 291: 1< 0 - - > 2< 1191936 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400729:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a886e6ef 00000080:00200000:0.0:1787827241.400730:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 292. 00000080:00200000:0.0:1787827241.400732:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 292 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.400746:0:48385:0:(rw26.c:875:ll_write_end()) pos 1196032, len 4096, copied 4096 00000080:00200000:0.0:1787827241.400752:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007a547316[3 00000000bdb38050 2 1 00000000746d8de4 0 0 292] 00000080:00200000:0.0:1787827241.400755:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000dd085704 17ffffc0001001 3:0 ffff9a788a7e53a0 292 lru 00000080:00200000:0.0:1787827241.400762:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000008c4a31 292: 1< 0 - - > 2< 1196032 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400764:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007a547316 00000080:00200000:0.0:1787827241.400766:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 293. 00000080:00200000:0.0:1787827241.400767:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 293 of 0 to 742 bytes 00000080:00200000:0.0:1787827241.400789:0:48385:0:(rw26.c:875:ll_write_end()) pos 1200128, len 742, copied 742 00000080:00200000:0.0:1787827241.400792:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000099293af5[3 00000000bdb38050 2 1 00000000746d8de4 0 0 293] 00000080:00200000:0.0:1787827241.400794:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006127f0d9 17ffffc0001001 3:0 ffff9a788a7e50e8 293 lru 00000080:00200000:0.0:1787827241.400798:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000626d6270 293: 1< 0 - - > 2< 1200128 0 0 0x0 0x0 | 0000000023a82262 000000001c0019c3 > 3< 0 0 > 4< 0 0 8 2109440 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.400800:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000099293af5 00000080:00200000:0.0:1787827241.400801:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 294. 00000080:00200000:0.0:1787827241.400804:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 294, from 0, to 742 prio 0 00000008:00010000:0.0:1787827241.400858:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400879:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400891:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400897:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400902:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400907:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400912:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400917:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400922:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400926:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400931:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400938:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400943:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400948:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400953:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.400964:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.400969:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400975:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400979:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400984:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400994:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.400999:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401004:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401009:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401014:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401019:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401023:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401028:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401033:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401039:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401045:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401052:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401055:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401060:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401065:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401070:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401075:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401080:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401095:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401105:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401116:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401126:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401136:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401142:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401147:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401152:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401157:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401166:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401170:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401176:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401181:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401186:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401191:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401196:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401200:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401206:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401215:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401220:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401225:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401230:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401235:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401239:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401244:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401252:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401254:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401264:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401269:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401274:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401279:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401284:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401289:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401294:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401298:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401303:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401308:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401313:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401318:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401323:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401328:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401335:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401338:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401343:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401348:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401355:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401360:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401365:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401370:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401375:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401379:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401384:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401389:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401394:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401399:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401405:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401409:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401416:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401419:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401424:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401429:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401434:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401439:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401447:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401451:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401456:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401461:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401466:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401471:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401476:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401481:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401486:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401491:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401498:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401524:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401533:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401538:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401543:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401548:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401553:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401558:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401567:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401572:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401577:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401582:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401587:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401592:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401597:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401602:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401609:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401613:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401619:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401624:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401629:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401634:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401639:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401644:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401649:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401654:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401663:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401668:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401673:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401678:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401682:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401687:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401694:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401697:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401702:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401711:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401715:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401722:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401730:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401740:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401748:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401759:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401770:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401777:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401783:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401788:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401793:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401798:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401806:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401809:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401814:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401819:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401824:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401850:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401857:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401863:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401867:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401872:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401877:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401882:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401887:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401892:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401898:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401903:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401911:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401914:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401919:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401924:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401930:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401935:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401940:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401949:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401954:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401959:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401964:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401969:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401974:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401979:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401983:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.401988:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.401996:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.401999:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402005:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402010:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402014:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402019:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402024:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402029:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402034:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402043:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402049:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402053:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402058:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402063:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402068:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402073:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402079:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.402085:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402100:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402111:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402121:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402132:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402139:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402144:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402149:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402154:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402159:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402165:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402170:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402175:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402180:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402185:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402193:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.402196:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402201:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402206:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402215:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402221:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402226:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402230:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402235:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402240:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402245:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402250:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402255:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402261:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402267:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402272:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402278:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.402281:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402286:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402291:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402296:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402301:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402310:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402315:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402320:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402325:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402330:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402335:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402339:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402344:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402349:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402355:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402363:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.402365:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402370:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402375:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402380:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402385:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402390:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402395:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402403:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402408:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402413:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402417:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402422:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402427:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402432:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402437:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402443:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.402447:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402452:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402456:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402461:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402466:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402471:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402476:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402481:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402485:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402494:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402498:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402520:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402528:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402537:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402545:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402557:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.402563:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402574:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402586:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402592:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402597:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402602:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402606:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402611:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402616:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402621:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402626:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402632:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402638:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402642:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402647:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402655:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.402658:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402663:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402667:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402672:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402682:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402688:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402693:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402698:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.402702:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000da2aea7c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.402709:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcccb8, count 9, dirtied 9 00000080:00200000:0.0:1787827241.402711:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 294 pages 1200870 bytes, tot: 1200870 00000080:00200000:0.0:1787827241.402714:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) sanity.sh: write bytes 0, result: 1200870 00000020:00010000:0.0:1787827241.402732:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.402736:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.402740:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3ccb0 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.402749:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.402760:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.402779:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.402781:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3ccb0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.402790:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 1200870 bytes 00000080:00200000:0.0:1787827241.402797:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 293] 00000080:00200000:0.0:1787827241.402805:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x47:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.402808:0:48385:0:(file.c:2240:ll_file_io_generic()) sanity.sh: 2 io complete with rc: 0, result: 1200870, restart: 0 00000080:00200000:0.0:1787827241.402814:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x47:0x0] updating i_size 1200870 i_blocks 0 00000080:00200000:0.0:1787827241.402818:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 1200870 00000080:00200008:0.0:1787827241.402873:0:48385:0:(file.c:479:ll_file_release()) START file sanity.sh:[0x240001b71:0x47:0x0](ffff9a789dfccb08), flags 100001 00000002:00100000:0.0:1787827241.402926:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6530780 x1874671476638848/t17179869513(17179869513) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.402937:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.402941:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65330c0 x1874671476639488/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.402960:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65330c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639488:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.403027:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.404148:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65330c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639488:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.404166:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6530780 x1874671476638848/t17179869513(17179869513) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.404215:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.404228:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.404244:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.404257:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.404264:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.404269:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.404272:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.404277:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.404279:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.404284:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.404292:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.404297:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.404301:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.404304:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.404307:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.404310:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=sanity.sh, flags=8 00000080:00200008:0.0:1787827241.404316:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sanity.sh:[0x240001b71:0x47:0x0](ffff9a789dfccb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.404319:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x47:0x0](ffff9a789dfccb08),name=sanity.sh 00000002:00010000:0.0:1787827241.404329:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x47:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.404336:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.404342:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.404347:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.404350:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x47:0x0](ffff9a789dfccb08) 00000080:00010000:0.0:1787827241.404351:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x47:0x0](ffff9a789dfccb08) for lock 0xebdcd48a4298ee3d 00000080:00010000:0.0:1787827241.404354:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee3d from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.404355:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.404360:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.404375:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x47:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.404383:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x47:0x0] 00000008:00010000:0.0:1787827241.404394:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.404396:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f330, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.404407:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.404412:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.404420:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.404423:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.404429:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000035bd4b3/0xebdcd48a4298ee4b lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xa4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcae0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.404439:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.404443:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x47:0x0] updating i_size 1200870 i_blocks 0 00000020:00010000:0.0:1787827241.404447:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.404449:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.404451:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f330 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.404453:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.404455:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f330 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.404460:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x47:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.404498:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.404519:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.404525:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.404531:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.404536:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.404539:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.404541:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.404544:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.404546:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.404552:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.404562:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.404565:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.404567:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.404569:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.404571:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.404579:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=scrub-performance.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.404583:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=scrub-performance.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.404591:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: scrub-performance.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.404619:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.404625:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.404633:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.404642:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639616:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.404672:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.405657:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639616:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.405667:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405676:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405681:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405687:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405692:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405697:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.405701:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405706:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405711:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405716:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.405722:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405727:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee52 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.405738:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6532d00 x1874671476639616/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.405748:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.405762:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.405790:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.405796:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.405802:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.405809:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.405814:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.405817:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.405819:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.405822:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.405824:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.405847:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.405855:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.405858:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.405861:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.405863:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.405865:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.405871:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=scrub-performance.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:0.0:1787827241.405874:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=scrub-performance.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.405880:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: scrub-performance.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.405896:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405901:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.405918:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.405928:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639744:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.405957:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.406788:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476639744:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.406796:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406802:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406806:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406811:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406815:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406818:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04a00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.406820:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406824:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406843:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406849:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.406852:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.406857:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298ee59 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.406865:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6532d00 x1874671476639744/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.406872:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.406880:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.406920:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.406925:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.406930:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.406937:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.406942:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.406944:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.406946:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.406949:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:0.0:1787827241.406952:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.406957:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.406964:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.406967:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.406969:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.406972:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.406974:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.406980:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.406984:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=scrub-performance.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78a6580600, open_flags 80c1, mode 81a4 opened 0 00000080:00200000:0.0:1787827241.406989:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=scrub-performance.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:0.0:1787827241.407002:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: scrub-performance.sh,[0x240001b71:0x48:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:0.0:1787827241.407024:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.407030:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827241.407036:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827241.407040:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6532d00 x1874671476640000/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827241.407046:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.407055:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640000:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.407152:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.408736:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640000:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.408748:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408764:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408770:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408779:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408788:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408794:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408799:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408804:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.408811:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6532d00 x1874671476640000/t17179869515(17179869515) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827241.408820:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6532d00 x1874671476640000/t17179869515(17179869515) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827241.408847:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827241.408856:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.408863:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/72 (0 0) 00000080:00200000:0.0:1787827241.408867:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827241.408873:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=scrub-performance.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:0.0:1787827241.408897:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x48:0x0](ffff9a789dfce3c8) 00000080:00200000:0.0:1787827241.408963:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x48:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787827241.408967:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x48:0x0](ffff9a789dfce3c8) 00000080:00010000:0.0:1787827241.408969:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x48:0x0](ffff9a789dfce3c8) for lock 0xebdcd48a4298ee60 00000080:00010000:0.0:1787827241.408975:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x48:0x0](ffff9a789dfce3c8) 00000080:00200008:0.0:1787827241.408982:0:48385:0:(file.c:1090:ll_file_open()) START file scrub-performance.sh:[0x240001b71:0x48:0x0](ffff9a789dfce3c8), flags 100301 00000002:00100000:0.0:1787827241.408995:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6532d00 x1874671476640000/t17179869515(17179869515) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827241.409025:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee60 from it ffff9a7887be0720 00010000:00010000:0.0:1787827241.409028:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.409035:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.409046:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x48:0x0](ffff9a789dfce3c8), inode mode 81a4 mask 40 00000080:00200008:0.0:1787827241.409055:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file scrub-performance.sh:[0x240001b71:0x48:0x0](ffff9a789dfce3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.409058:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x48:0x0](ffff9a789dfce3c8),name=scrub-performance.sh 00000002:00010000:0.0:1787827241.409066:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x48:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.409072:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.409078:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.409083:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.409085:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x48:0x0](ffff9a789dfce3c8) 00000080:00010000:0.0:1787827241.409087:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x48:0x0](ffff9a789dfce3c8) for lock 0xebdcd48a4298ee60 00000080:00010000:0.0:1787827241.409089:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee60 from it ffffaa5487e7bcd8 00010000:00010000:0.0:1787827241.409091:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.409097:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.409105:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x48:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.409110:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x48:0x0] 00000008:00010000:0.0:1787827241.409118:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.409119:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f630, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.409142:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000457495c/0xebdcd48a4298ee67 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.409149:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000457495c/0xebdcd48a4298ee67 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.409178:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:0.0:1787827241.410226:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x48:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1787827241.410232:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.410240:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.410245:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f630 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.410248:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.410250:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f630 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.410259:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x48:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:0.0:1787827241.410286:0:48385:0:(file.c:2652:do_file_write_iter()) START file scrub-performance.sh:[0x240001b71:0x48:0x0], ppos: 0, count: 5105 00000080:00200000:0.0:1787827241.410293:0:48385:0:(file.c:2090:ll_file_io_generic()) scrub-performance.sh: write ppos: 0, bytes: 5105 00000080:00200000:0.0:1787827241.410298:0:48385:0:(file.c:1848:ll_io_set_mirror()) scrub-performance.sh: desiginated mirror: 0 00000020:00200000:0.0:1787827241.410303:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@0000000077a8ccc8[0x110, 1, [0x240001b71:0x48:0x0] hash] 00000020:00200000:0.0:1787827241.410305:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 5105) 0 0 00000080:00200000:0.0:1787827241.410307:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x48:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827241.410323:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 1] 00020000:00200000:0.0:1787827241.410333:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 5105) 5105, 5105 00020000:00200000:0.0:1787827241.410337:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787827241.410348:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 1] 00000008:00010000:0.0:1787827241.410355:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:0.0:1787827241.410358:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f630, osc lock ffff9a7886bc5380, flags 0x0 00010000:00010000:0.0:1787827241.410365:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/165 (0 8191) 00010000:00010000:0.0:1787827241.410382:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.410389:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.410395:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.410405:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531680 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640256:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.410435:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.411149:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.411204:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786560 00000100:00100000:1.0:1787827241.411213:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786560 00000100:00100000:1.0:1787827241.411269:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 297 00000100:00100000:1.0:1787827241.411282:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6532940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488786560:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.411296:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000457495c/0xebdcd48a4298ee67 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcb34 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.411357:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.411368:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6532940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488786560:12345-192.168.203.140@tcp:104: Request processed in 87us (217us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.411378:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 297 00000100:00100000:1.0:1787827241.411547:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838cad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476640384:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.411960:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531680 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640256:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.411973:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.411981:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.411990:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.412346:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:2.0:1787827241.412383:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786688 00000100:00100000:2.0:1787827241.412388:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786688 00000100:00100000:2.0:1787827241.412416:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 298 00000100:00100000:2.0:1787827241.412423:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6533840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488786688:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.412429:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.412437:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.412445:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.412452:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.412458:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.412464:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.412481:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7885a04200) 00010000:00010000:0.0:1787827241.412495:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.412514:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.412524:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.412524:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6533840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488786688:12345-192.168.203.140@tcp:105: Request processed in 101us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.412536:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 298 00000008:00010000:0.0:1787827241.412541:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,2 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.412550:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.412557:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.412573:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) scrub-performance.sh: write [0, 5105) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.412584:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.412617:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827241.412627:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000da68121[3 00000000ebf0945a 2 1 00000000746d8de4 0 0 0] 00000080:00200000:0.0:1787827241.412632:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cccda3b7 17ffffc0001001 3:0 ffff9a788a7e52b8 0 lru 00000080:00200000:0.0:1787827241.412639:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a3f2d9cb 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 00000000ce6482d6 > 3< 0 0 > 4< 0 0 8 1695744 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.412642:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000da68121 00000080:00200000:0.0:1787827241.412644:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.412650:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 1009 bytes 00000080:00200000:0.0:1787827241.412683:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 1009, copied 1009 00000080:00200000:0.0:1787827241.412687:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005b1e8113[3 00000000ebf0945a 2 1 00000000746d8de4 0 0 1] 00000080:00200000:0.0:1787827241.412690:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c677eac0 17ffffc0001021 2:0 ffff9a788a7e51d0 1 lru 00000080:00200000:0.0:1787827241.412694:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000094f0ce50 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000043fddcb7 00000000ce6482d6 > 3< 0 0 > 4< 0 0 8 1695744 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.412696:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005b1e8113 00000080:00200000:0.0:1787827241.412697:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:0.0:1787827241.412700:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 2, from 0, to 1009 prio 0 00000008:00010000:0.0:1787827241.412717:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 000000000bdb0af2 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.412732:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000000bdb0af2 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.412741:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfce578, count 2, dirtied 2 00000080:00200000:0.0:1787827241.412744:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 2 pages 5105 bytes, tot: 5105 00000080:00200000:0.0:1787827241.412747:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) scrub-performance.sh: write bytes 0, result: 5105 00000020:00010000:0.0:1787827241.412757:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.412759:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.412761:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f630 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.412765:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.412771:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.412779:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.412781:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f630 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.412786:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 5105 bytes 00000080:00200000:0.0:1787827241.412791:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 1] 00000080:00200000:0.0:1787827241.412795:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x48:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.412798:0:48385:0:(file.c:2240:ll_file_io_generic()) scrub-performance.sh: 2 io complete with rc: 0, result: 5105, restart: 0 00000080:00200000:0.0:1787827241.412802:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x48:0x0] updating i_size 5105 i_blocks 0 00000080:00200000:0.0:1787827241.412805:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 5105 00000080:00200008:0.0:1787827241.412820:0:48385:0:(file.c:479:ll_file_release()) START file scrub-performance.sh:[0x240001b71:0x48:0x0](ffff9a789dfce3c8), flags 100001 00000002:00100000:0.0:1787827241.412880:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6532d00 x1874671476640000/t17179869515(17179869515) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.412890:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.412894:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6531680 x1874671476640640/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.412908:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531680 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640640:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.412948:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.413223:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838cad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476640384:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787827241.413743:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531680 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640640:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.413760:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6532d00 x1874671476640000/t17179869515(17179869515) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.413792:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.413800:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.413807:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.413815:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.413821:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.413824:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.413826:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.413864:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.413869:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.413879:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.413890:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.413894:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.413898:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.413900:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.413902:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.413905:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=scrub-performance.sh, flags=8 00000080:00200008:0.0:1787827241.413909:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file scrub-performance.sh:[0x240001b71:0x48:0x0](ffff9a789dfce3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.413912:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x48:0x0](ffff9a789dfce3c8),name=scrub-performance.sh 00000002:00010000:0.0:1787827241.413921:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x48:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.413926:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.413933:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.413938:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.413940:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x48:0x0](ffff9a789dfce3c8) 00000080:00010000:0.0:1787827241.413942:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x48:0x0](ffff9a789dfce3c8) for lock 0xebdcd48a4298ee60 00000080:00010000:0.0:1787827241.413945:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee60 from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.413947:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.413956:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.413965:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x48:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.413969:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x48:0x0] 00000008:00010000:0.0:1787827241.413975:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.413977:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f630, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.413984:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.413991:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.413997:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.414000:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.414005:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003ffef192/0xebdcd48a4298ee6e lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb3b expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.414010:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.414013:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x48:0x0] updating i_size 5105 i_blocks 0 00000020:00010000:0.0:1787827241.414016:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.414018:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.414019:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f630 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.414022:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.414024:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f630 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.414029:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x48:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.414056:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.414061:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.414066:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.414072:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.414077:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.414079:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.414081:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.414084:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.414086:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.414092:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.414097:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.414099:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.414101:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.414103:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.414104:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.414110:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=setup-cifs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.414113:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup-cifs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.414119:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup-cifs.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.414139:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.414144:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.414150:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.414159:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640768:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.414189:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.415103:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640768:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.415114:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415123:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415131:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415139:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415148:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415154:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.415159:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415167:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415172:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415177:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.415180:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415185:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ee75 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.415194:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6530b40 x1874671476640768/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.415203:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.415213:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.415233:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.415239:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.415244:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.415251:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.415256:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.415259:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.415261:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.415265:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.415267:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.415273:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.415278:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.415281:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.415284:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.415286:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.415287:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.415293:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=setup-cifs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:0.0:1787827241.415295:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup-cifs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.415301:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup-cifs.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.415315:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415320:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.415325:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.415334:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640896:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.415365:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.416225:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476640896:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.416232:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416239:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416244:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416248:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416252:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416256:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a05800 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.416258:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416261:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416265:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416269:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.416271:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416276:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298ee7c lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.416284:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6530b40 x1874671476640896/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.416295:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.416303:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.416338:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.416344:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.416349:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.416355:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.416360:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.416363:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.416365:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.416368:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:0.0:1787827241.416370:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.416375:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.416381:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.416384:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.416386:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.416388:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.416390:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.416395:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.416399:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=setup-cifs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78a6581b00, open_flags 80c1, mode 81a4 opened 0 00000080:00200000:0.0:1787827241.416403:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup-cifs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:0.0:1787827241.416413:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup-cifs.sh,[0x240001b71:0x49:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:0.0:1787827241.416436:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.416444:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827241.416453:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827241.416459:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6530b40 x1874671476641152/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827241.416469:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.416479:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641152:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.416572:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.417926:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530b40 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641152:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.417935:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.417946:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.417951:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.417960:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.417977:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.417987:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.417996:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.418005:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.418015:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6530b40 x1874671476641152/t17179869517(17179869517) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827241.418025:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6530b40 x1874671476641152/t17179869517(17179869517) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827241.418038:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827241.418049:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.418059:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/73 (0 0) 00000080:00200000:0.0:1787827241.418064:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827241.418071:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=setup-cifs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:0.0:1787827241.418097:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x49:0x0](ffff9a789dfc8908) 00000080:00200000:0.0:1787827241.418160:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x49:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787827241.418165:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x49:0x0](ffff9a789dfc8908) 00000080:00010000:0.0:1787827241.418168:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x49:0x0](ffff9a789dfc8908) for lock 0xebdcd48a4298ee83 00000080:00010000:0.0:1787827241.418174:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x49:0x0](ffff9a789dfc8908) 00000080:00200008:0.0:1787827241.418180:0:48385:0:(file.c:1090:ll_file_open()) START file setup-cifs.sh:[0x240001b71:0x49:0x0](ffff9a789dfc8908), flags 100301 00000002:00100000:0.0:1787827241.418193:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6530b40 x1874671476641152/t17179869517(17179869517) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827241.418218:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee83 from it ffff9a7887be0720 00010000:00010000:0.0:1787827241.418223:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.418232:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.418246:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x49:0x0](ffff9a789dfc8908), inode mode 81a4 mask 40 00000080:00200008:0.0:1787827241.418258:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file setup-cifs.sh:[0x240001b71:0x49:0x0](ffff9a789dfc8908), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.418264:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x49:0x0](ffff9a789dfc8908),name=setup-cifs.sh 00000002:00010000:0.0:1787827241.418275:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x49:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.418281:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.418288:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.418293:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.418296:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x49:0x0](ffff9a789dfc8908) 00000080:00010000:0.0:1787827241.418298:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x49:0x0](ffff9a789dfc8908) for lock 0xebdcd48a4298ee83 00000080:00010000:0.0:1787827241.418301:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee83 from it ffffaa5487e7bcd8 00010000:00010000:0.0:1787827241.418304:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.418309:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.418319:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x49:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.418324:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x49:0x0] 00000008:00010000:0.0:1787827241.418332:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.418334:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f3b0, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.418355:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.418362:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.418389:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.418428:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476641280:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.419250:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.419267:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.419282:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.419294:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.419306:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 5/2,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.419316:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.419328:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.419350:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x49:0x0] updating i_size 0 i_blocks 0 00010000:00010000:1.0:1787827241.419351:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00010000:0.0:1787827241.419355:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.419361:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.419366:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f3b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:1.0:1787827241.419370:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00010000:0.0:1787827241.419371:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.419377:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f3b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.419389:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x49:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:1.0:1787827241.419390:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.419407:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476641280:192.168.203.140@tcp:101:cp.0 00000080:00200008:0.0:1787827241.419416:0:48385:0:(file.c:2652:do_file_write_iter()) START file setup-cifs.sh:[0x240001b71:0x49:0x0], ppos: 0, count: 2475 00000080:00200000:0.0:1787827241.419428:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 2475 bytes 00000080:00200000:0.0:1787827241.419433:0:48385:0:(file.c:2628:ll_do_tiny_write()) result: 0, original count 2475 00000080:00200000:0.0:1787827241.419438:0:48385:0:(file.c:2090:ll_file_io_generic()) setup-cifs.sh: write ppos: 0, bytes: 2475 00000080:00200000:0.0:1787827241.419444:0:48385:0:(file.c:1848:ll_io_set_mirror()) setup-cifs.sh: desiginated mirror: 0 00000020:00200000:0.0:1787827241.419450:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000baca4d0f[0x110, 1, [0x240001b71:0x49:0x0] hash] 00000020:00200000:0.0:1787827241.419452:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 2475) 0 0 00000080:00200000:0.0:1787827241.419455:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x49:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827241.419463:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 0] 00020000:00200000:0.0:1787827241.419469:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 2475) 2475, 2475 00020000:00200000:0.0:1787827241.419471:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787827241.419480:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 0] 00000008:00010000:0.0:1787827241.419486:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:0.0:1787827241.419489:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f3b0, osc lock ffff9a7886bc5380, flags 0x0 00010000:00010000:0.0:1787827241.419495:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419264/165 (0 4095) 00010000:00010000:0.0:1787827241.419522:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.419529:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.419535:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.419545:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641408:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.419582:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.420274:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.420331:0:48371:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787827241.420344:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786816 00000100:00100000:1.0:1787827241.420349:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786816 00000100:00100000:1.0:1787827241.420390:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 299 00000100:00100000:1.0:1787827241.420399:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6439850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488786816:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.420411:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.420458:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.420466:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6439850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488786816:12345-192.168.203.140@tcp:104: Request processed in 68us (193us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.420476:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 299 00010000:00010000:2.0:1787827241.420488:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827241.420496:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.420516:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04a00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:2.0:1787827241.420520:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.420532:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x49:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.420540:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x49:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827241.420567:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.420573:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.420591:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.420602:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ee8a lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcb8f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1787827241.420614:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476641536:192.168.203.140@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787827241.420617:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.420620:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.420624:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.421005:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641408:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.421018:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421029:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421039:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.421354:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488786944 00000100:00100000:2.0:1787827241.421360:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488786944 00000100:00100000:2.0:1787827241.421400:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 300 00000100:00100000:2.0:1787827241.421416:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838cad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75570:x1874671488786944:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.421428:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421438:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421447:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421457:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421466:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421474:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.421496:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7885a04600) 00010000:00010000:0.0:1787827241.421523:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.421534:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.421544:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838cad00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75570:x1874671488786944:12345-192.168.203.140@tcp:105: Request processed in 134us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.421558:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 300 00010000:00010000:0.0:1787827241.421559:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.421571:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,2 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.421579:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.421586:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.421600:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) setup-cifs.sh: write [0, 2475) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.421608:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 2475 bytes 00000080:00200000:0.0:1787827241.421638:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 2475, copied 2475 00000080:00200000:0.0:1787827241.421645:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000bd7f74c0[3 0000000079fff55f 2 1 00000000746d8de4 0 0 0] 00000080:00200000:0.0:1787827241.421648:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000028bef1aa 17ffffc0001001 3:0 ffff9a788a7e5740 0 lru 00000080:00200000:0.0:1787827241.421653:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002dc03cfb 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 00000000a40683bd > 3< 0 0 > 4< 0 0 8 880640 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.421656:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000bd7f74c0 00000080:00200000:0.0:1787827241.421657:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.421660:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 1, from 0, to 2475 prio 0 00000008:00010000:0.0:1787827241.421678:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000666665c1 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.421691:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc8ab8, count 1, dirtied 1 00000080:00200000:0.0:1787827241.421695:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 1 pages 2475 bytes, tot: 2475 00000080:00200000:0.0:1787827241.421697:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) setup-cifs.sh: write bytes 0, result: 2475 00000020:00010000:0.0:1787827241.421707:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.421709:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.421712:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f3b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.421715:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.421724:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.421737:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.421740:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f3b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.421748:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 2475 bytes 00000080:00200000:0.0:1787827241.421754:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 0] 00000080:00200000:0.0:1787827241.421759:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x49:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.421764:0:48385:0:(file.c:2240:ll_file_io_generic()) setup-cifs.sh: 2 io complete with rc: 0, result: 2475, restart: 0 00000080:00200000:0.0:1787827241.421771:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x49:0x0] updating i_size 2475 i_blocks 0 00000080:00200000:0.0:1787827241.421775:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 2475 00000080:00200008:0.0:1787827241.421793:0:48385:0:(file.c:479:ll_file_release()) START file setup-cifs.sh:[0x240001b71:0x49:0x0](ffff9a789dfc8908), flags 100001 00000002:00100000:0.0:1787827241.421824:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6530b40 x1874671476641152/t17179869517(17179869517) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.421863:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.421870:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6531e00 x1874671476641792/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.421887:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6531e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641792:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.421930:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.422176:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476641536:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787827241.422600:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6531e00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641792:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.422616:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6530b40 x1874671476641152/t17179869517(17179869517) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.422645:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.422653:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.422661:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.422669:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.422674:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.422677:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.422679:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.422683:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.422685:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.422691:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.422696:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.422700:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.422703:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.422705:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.422708:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.422710:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=setup-cifs.sh, flags=8 00000080:00200008:0.0:1787827241.422714:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file setup-cifs.sh:[0x240001b71:0x49:0x0](ffff9a789dfc8908), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.422717:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x49:0x0](ffff9a789dfc8908),name=setup-cifs.sh 00000002:00010000:0.0:1787827241.422723:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x49:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.422727:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.422733:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.422738:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.422740:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x49:0x0](ffff9a789dfc8908) 00000080:00010000:0.0:1787827241.422741:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x49:0x0](ffff9a789dfc8908) for lock 0xebdcd48a4298ee83 00000080:00010000:0.0:1787827241.422743:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ee83 from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.422745:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.422751:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.422759:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x49:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.422764:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x49:0x0] 00000008:00010000:0.0:1787827241.422769:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.422771:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f3b0, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.422778:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.422785:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.422796:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.422798:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.422804:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000d103ba4a/0xebdcd48a4298ee91 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xa5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcb96 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.422809:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.422813:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x49:0x0] updating i_size 2475 i_blocks 0 00000020:00010000:0.0:1787827241.422815:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.422817:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.422819:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f3b0 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.422821:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.422823:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f3b0 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.422827:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x49:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.422883:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.422890:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.422896:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.422902:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.422907:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.422910:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.422912:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.422916:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.422918:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.422923:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.422929:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.422932:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.422934:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.422936:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.422938:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.422945:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=setup_kerberos.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.422948:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup_kerberos.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.422954:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup_kerberos.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.422977:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.422985:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.422994:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.423009:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641920:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.423041:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.423875:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476641920:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.423883:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423889:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423894:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423899:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423903:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423906:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a05c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.423910:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423914:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423918:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423922:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.423924:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.423929:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ee98 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.423937:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6533480 x1874671476641920/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.423949:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.423958:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.423981:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.423989:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.423997:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.424007:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.424015:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.424019:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.424021:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.424025:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.424027:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.424033:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.424039:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.424041:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.424044:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.424046:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.424047:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.424053:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=setup_kerberos.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:0.0:1787827241.424056:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup_kerberos.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.424062:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup_kerberos.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.424077:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424082:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424087:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.424095:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642048:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.424122:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.424901:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642048:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.424910:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424916:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424921:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424925:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424930:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424933:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04e00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.424935:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424940:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424944:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424948:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.424950:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.424954:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04e00/0xebdcd48a4298ee9f lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.424963:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6533480 x1874671476642048/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.424972:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.424983:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.425024:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.425030:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.425036:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.425042:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.425048:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.425051:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.425052:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.425057:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:0.0:1787827241.425059:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.425065:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.425071:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.425073:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.425076:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.425077:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.425079:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.425088:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.425091:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=setup_kerberos.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78a6580600, open_flags 80c1, mode 81ed opened 0 00000080:00200000:0.0:1787827241.425095:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup_kerberos.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:0.0:1787827241.425105:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup_kerberos.sh,[0x240001b71:0x4a:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:0.0:1787827241.425124:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.425129:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827241.425134:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827241.425138:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6533480 x1874671476642304/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827241.425144:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.425152:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642304:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.425222:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4180 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.426414:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533480 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642304:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.426424:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426439:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426449:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426461:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426472:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426482:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426488:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426493:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.426500:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6533480 x1874671476642304/t17179869519(17179869519) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827241.426525:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6533480 x1874671476642304/t17179869519(17179869519) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827241.426535:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827241.426543:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426550:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/74 (0 0) 00000080:00200000:0.0:1787827241.426553:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827241.426559:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=setup_kerberos.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:0.0:1787827241.426576:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x4a:0x0](ffff9a789dfc9988) 00000080:00200000:0.0:1787827241.426617:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x4a:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787827241.426621:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x4a:0x0](ffff9a789dfc9988) 00000080:00010000:0.0:1787827241.426624:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4a:0x0](ffff9a789dfc9988) for lock 0xebdcd48a4298eea6 00000080:00010000:0.0:1787827241.426629:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4a:0x0](ffff9a789dfc9988) 00000080:00200008:0.0:1787827241.426635:0:48385:0:(file.c:1090:ll_file_open()) START file setup_kerberos.sh:[0x240001b71:0x4a:0x0](ffff9a789dfc9988), flags 100301 00000002:00100000:0.0:1787827241.426646:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6533480 x1874671476642304/t17179869519(17179869519) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827241.426670:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eea6 from it ffff9a7887be0720 00010000:00010000:0.0:1787827241.426674:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426681:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.426694:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4a:0x0](ffff9a789dfc9988), inode mode 81ed mask 40 00000080:00200008:0.0:1787827241.426703:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file setup_kerberos.sh:[0x240001b71:0x4a:0x0](ffff9a789dfc9988), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.426706:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4a:0x0](ffff9a789dfc9988),name=setup_kerberos.sh 00000002:00010000:0.0:1787827241.426714:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4a:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.426719:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.426726:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.426731:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.426733:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4a:0x0](ffff9a789dfc9988) 00000080:00010000:0.0:1787827241.426735:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4a:0x0](ffff9a789dfc9988) for lock 0xebdcd48a4298eea6 00000080:00010000:0.0:1787827241.426737:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eea6 from it ffffaa5487e7bcd8 00010000:00010000:0.0:1787827241.426740:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.426746:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.426754:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4a:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.426758:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4a:0x0] 00000008:00010000:0.0:1787827241.426765:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.426767:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f430, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.426788:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.426799:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.426824:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.426859:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476642432:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.427569:0:10934:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.427584:0:10934:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.427599:0:10934:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.427611:0:10934:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.427623:0:10934:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 5/2,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.427634:0:10934:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.427648:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.427658:0:10934:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.427667:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.427684:0:10934:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.427686:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4a:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1787827241.427694:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.427700:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.427704:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f430 (ffff9a78899db758/1) at cl_lock_cancel():142 00000100:00100000:1.0:1787827241.427706:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476642432:192.168.203.140@tcp:101:cp.0 00000020:00010000:0.0:1787827241.427710:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.427716:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f430 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.427728:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4a:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:0.0:1787827241.427766:0:48385:0:(file.c:2652:do_file_write_iter()) START file setup_kerberos.sh:[0x240001b71:0x4a:0x0], ppos: 0, count: 24288 00000080:00200000:0.0:1787827241.427774:0:48385:0:(file.c:2090:ll_file_io_generic()) setup_kerberos.sh: write ppos: 0, bytes: 24288 00000080:00200000:0.0:1787827241.427779:0:48385:0:(file.c:1848:ll_io_set_mirror()) setup_kerberos.sh: desiginated mirror: 0 00000020:00200000:0.0:1787827241.427785:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@000000008bf61f40[0x110, 1, [0x240001b71:0x4a:0x0] hash] 00000020:00200000:0.0:1787827241.427787:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 24288) 0 0 00000080:00200000:0.0:1787827241.427790:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4a:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827241.427798:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 5] 00020000:00200000:0.0:1787827241.427805:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 24288) 24288, 24288 00020000:00200000:0.0:1787827241.427808:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787827241.427818:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 5] 00000008:00010000:0.0:1787827241.427826:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:0.0:1787827241.427849:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f430, osc lock ffff9a7886bc5380, flags 0x0 00010000:00010000:0.0:1787827241.427858:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/166 (0 24575) 00010000:00010000:0.0:1787827241.427882:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.427893:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->24575] (req 0->24575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.427904:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->24575] (req 0->24575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.427930:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642560:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.427968:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.428682:0:10928:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:1.0:1787827241.428733:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787072 00000100:00100000:1.0:1787827241.428739:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787072 00000100:00100000:1.0:1787827241.428780:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 301 00000100:00100000:1.0:1787827241.428790:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6530000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488787072:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.428801:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.428905:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.428914:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6530000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488787072:12345-192.168.203.140@tcp:104: Request processed in 124us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.428924:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 301 00010000:00010000:0.0:1787827241.428927:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787827241.428936:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.428948:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a05e00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:0.0:1787827241.428954:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.428971:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827241.428980:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:0.0:1787827241.429007:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.429016:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787827241.429036:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.429046:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062f88089/0xebdcd48a4298eead lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcbea expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.429066:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:0.0:1787827241.429070:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827241.429075:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827241.429077:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827241.429079:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65312c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476642688:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.429632:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642560:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.429647:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->24575] (req 0->24575) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.429661:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->24575] (req 0->24575) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.429667:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->24575] (req 0->24575) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.429957:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787200 00000100:00100000:2.0:1787827241.429961:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787200 00000100:00100000:2.0:1787827241.429992:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 302 00000100:00100000:2.0:1787827241.430002:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c9680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:78039:x1874671488787200:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.430011:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->24575] (req 0->24575) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.430022:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->24575] (req 0->24575) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.430032:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.430042:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.430053:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.430069:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.430088:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7885a04400) 00000100:00100000:2.0:1787827241.430113:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c9680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:78039:x1874671488787200:12345-192.168.203.140@tcp:105: Request processed in 115us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.430119:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 302 00010000:00010000:0.0:1787827241.430153:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.430167:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.430176:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.430184:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,2 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.430191:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.430198:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.430213:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) setup_kerberos.sh: write [0, 24288) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.430222:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.430260:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827241.430267:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000168370ea[3 00000000715d1e7b 2 1 00000000746d8de4 0 0 0] 00000080:00200000:0.0:1787827241.430270:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000195da4e9 17ffffc0001001 3:0 ffff9a788a7e5e80 0 lru 00000080:00200000:0.0:1787827241.430276:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a9058c9b 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 000000004d12abc6 > 3< 0 0 > 4< 0 0 8 1662976 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.430279:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000168370ea 00000080:00200000:0.0:1787827241.430281:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.430283:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.430299:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:0.0:1787827241.430302:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004164972a[3 00000000715d1e7b 2 1 00000000746d8de4 0 0 1] 00000080:00200000:0.0:1787827241.430304:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b3b55b0c 17ffffc0001001 3:0 ffff9a788a7e5488 1 lru 00000080:00200000:0.0:1787827241.430308:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000080c4f6e6 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000043fddcb7 000000004d12abc6 > 3< 0 0 > 4< 0 0 8 1662976 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.430310:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004164972a 00000080:00200000:0.0:1787827241.430311:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:0.0:1787827241.430312:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.430322:0:48385:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:0.0:1787827241.430324:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006f9368be[3 00000000715d1e7b 2 1 00000000746d8de4 0 0 2] 00000080:00200000:0.0:1787827241.430326:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ab8a4f44 17ffffc0001001 3:0 ffff9a788a7e5bc8 2 lru 00000080:00200000:0.0:1787827241.430329:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f9a4adb7 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000043fddcb7 000000004d12abc6 > 3< 0 0 > 4< 0 0 8 1662976 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.430331:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006f9368be 00000080:00200000:0.0:1787827241.430332:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:0.0:1787827241.430333:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.430342:0:48385:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:0.0:1787827241.430345:0:48385:0:(rw26.c:933:ll_write_end()) page@000000000c0ff94a[3 00000000715d1e7b 2 1 00000000746d8de4 0 0 3] 00000080:00200000:0.0:1787827241.430347:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000063412fac 17ffffc0001001 3:0 ffff9a788a7e5570 3 lru 00000080:00200000:0.0:1787827241.430351:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000cb5ee605 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000043fddcb7 000000004d12abc6 > 3< 0 0 > 4< 0 0 8 1662976 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.430353:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000000c0ff94a 00000080:00200000:0.0:1787827241.430353:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:0.0:1787827241.430354:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.430365:0:48385:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:0.0:1787827241.430367:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000005f1b693[3 00000000715d1e7b 2 1 00000000746d8de4 0 0 4] 00000080:00200000:0.0:1787827241.430369:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000831e726d 17ffffc0001001 3:0 ffff9a788a7e5828 4 lru 00000080:00200000:0.0:1787827241.430373:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000372667bc 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000043fddcb7 000000004d12abc6 > 3< 0 0 > 4< 0 0 8 1662976 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.430374:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000005f1b693 00000080:00200000:0.0:1787827241.430375:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:0.0:1787827241.430376:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 3808 bytes 00000080:00200000:0.0:1787827241.430388:0:48385:0:(rw26.c:875:ll_write_end()) pos 20480, len 3808, copied 3808 00000080:00200000:0.0:1787827241.430391:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000da9b625e[3 00000000715d1e7b 2 1 00000000746d8de4 0 0 5] 00000080:00200000:0.0:1787827241.430393:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000049b2bcb8 17ffffc0001001 3:0 ffff9a788a7e5000 5 lru 00000080:00200000:0.0:1787827241.430396:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003b9c08b4 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000043fddcb7 000000004d12abc6 > 3< 0 0 > 4< 0 0 8 1662976 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.430398:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000da9b625e 00000080:00200000:0.0:1787827241.430399:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:0.0:1787827241.430401:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 6, from 0, to 3808 prio 0 00000008:00010000:0.0:1787827241.430418:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 000000008af8aeeb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.430434:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008af8aeeb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.430444:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008af8aeeb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.430455:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008af8aeeb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.430466:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008af8aeeb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.430476:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000008af8aeeb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.430490:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfc9b38, count 6, dirtied 6 00000080:00200000:0.0:1787827241.430495:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 6 pages 24288 bytes, tot: 24288 00000080:00200000:0.0:1787827241.430499:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) setup_kerberos.sh: write bytes 0, result: 24288 00000020:00010000:0.0:1787827241.430535:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.430538:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.430540:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f430 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.430543:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.430554:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.430565:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.430568:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f430 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.430576:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 24288 bytes 00000080:00200000:0.0:1787827241.430581:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 5] 00000080:00200000:0.0:1787827241.430586:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4a:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.430591:0:48385:0:(file.c:2240:ll_file_io_generic()) setup_kerberos.sh: 2 io complete with rc: 0, result: 24288, restart: 0 00000080:00200000:0.0:1787827241.430598:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4a:0x0] updating i_size 24288 i_blocks 0 00000080:00200000:0.0:1787827241.430602:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 24288 00000080:00200008:0.0:1787827241.430619:0:48385:0:(file.c:479:ll_file_release()) START file setup_kerberos.sh:[0x240001b71:0x4a:0x0](ffff9a789dfc9988), flags 100001 00000002:00100000:0.0:1787827241.430644:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6533480 x1874671476642304/t17179869519(17179869519) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.430653:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.430657:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6533c00 x1874671476642944/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.430669:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642944:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.430702:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4300 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.430710:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65312c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476642688:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787827241.431459:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533c00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476642944:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.431486:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6533480 x1874671476642304/t17179869519(17179869519) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.431533:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.431544:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.431552:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.431560:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.431566:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.431570:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.431572:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.431576:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.431578:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.431584:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.431591:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.431594:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.431597:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.431600:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.431602:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.431604:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=setup_kerberos.sh, flags=8 00000080:00200008:0.0:1787827241.431608:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file setup_kerberos.sh:[0x240001b71:0x4a:0x0](ffff9a789dfc9988), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.431611:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4a:0x0](ffff9a789dfc9988),name=setup_kerberos.sh 00000002:00010000:0.0:1787827241.431617:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4a:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.431622:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.431628:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.431633:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.431635:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4a:0x0](ffff9a789dfc9988) 00000080:00010000:0.0:1787827241.431636:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4a:0x0](ffff9a789dfc9988) for lock 0xebdcd48a4298eea6 00000080:00010000:0.0:1787827241.431638:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eea6 from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.431640:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.431646:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.431655:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4a:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.431660:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4a:0x0] 00000008:00010000:0.0:1787827241.431666:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.431668:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f430, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.431675:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.431682:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.431690:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.431692:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.431698:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003070f57b/0xebdcd48a4298eeb4 lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->24575) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcbf1 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.431704:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.431707:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4a:0x0] updating i_size 24288 i_blocks 0 00000020:00010000:0.0:1787827241.431710:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.431712:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.431714:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f430 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.431715:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.431717:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f430 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.431722:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4a:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.431747:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.431753:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.431757:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.431763:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.431768:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.431771:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.431773:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.431776:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.431778:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.431788:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.431793:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.431796:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.431798:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.431800:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.431801:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.431808:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=setup-nfs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.431811:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup-nfs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.431817:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup-nfs.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.431852:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.431858:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.431864:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.431874:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643072:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.431905:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.432905:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643072:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.432912:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432918:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432923:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432928:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432932:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432936:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a05800 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.432939:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432943:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432947:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432951:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.432953:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.432958:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05800/0xebdcd48a4298eebb lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.432967:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a65321c0 x1874671476643072/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.432975:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.432984:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.433004:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.433009:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.433014:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.433020:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.433025:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.433027:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.433029:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.433032:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.433034:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.433039:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.433045:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.433048:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.433050:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.433052:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.433054:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.433059:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=setup-nfs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:0.0:1787827241.433061:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup-nfs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.433066:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup-nfs.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.433079:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.433088:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.433093:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.433101:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643200:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.433136:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.434023:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643200:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.434031:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434037:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434041:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434045:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434050:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434053:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.434055:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434059:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434063:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434067:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.434070:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434074:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298eec2 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.434081:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a65321c0 x1874671476643200/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.434087:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.434095:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.434135:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.434144:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.434151:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.434162:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.434170:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.434174:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.434175:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.434179:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:0.0:1787827241.434181:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.434191:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.434198:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.434200:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.434203:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.434205:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.434206:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.434212:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.434216:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=setup-nfs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78a6581e00, open_flags 80c1, mode 81ed opened 0 00000080:00200000:0.0:1787827241.434219:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=setup-nfs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:0.0:1787827241.434231:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: setup-nfs.sh,[0x240001b71:0x4b:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:0.0:1787827241.434250:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.434255:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827241.434261:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827241.434265:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65321c0 x1874671476643456/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827241.434270:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.434279:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643456:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.434356:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78908c4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.435601:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65321c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643456:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.435614:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435630:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435639:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435651:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435662:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435675:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435684:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435693:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.435704:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a65321c0 x1874671476643456/t17179869521(17179869521) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827241.435718:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a65321c0 x1874671476643456/t17179869521(17179869521) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827241.435730:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827241.435742:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435752:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/75 (0 0) 00000080:00200000:0.0:1787827241.435756:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827241.435765:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=setup-nfs.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:0.0:1787827241.435788:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x4b:0x0](ffff9a789dfcaa08) 00000080:00200000:0.0:1787827241.435869:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x4b:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787827241.435875:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x4b:0x0](ffff9a789dfcaa08) 00000080:00010000:0.0:1787827241.435878:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4b:0x0](ffff9a789dfcaa08) for lock 0xebdcd48a4298eec9 00000080:00010000:0.0:1787827241.435886:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4b:0x0](ffff9a789dfcaa08) 00000080:00200008:0.0:1787827241.435894:0:48385:0:(file.c:1090:ll_file_open()) START file setup-nfs.sh:[0x240001b71:0x4b:0x0](ffff9a789dfcaa08), flags 100301 00000002:00100000:0.0:1787827241.435912:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a65321c0 x1874671476643456/t17179869521(17179869521) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827241.435945:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eec9 from it ffff9a7887be0720 00010000:00010000:0.0:1787827241.435950:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.435960:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.435974:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4b:0x0](ffff9a789dfcaa08), inode mode 81ed mask 40 00000080:00200008:0.0:1787827241.435987:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file setup-nfs.sh:[0x240001b71:0x4b:0x0](ffff9a789dfcaa08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.435993:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4b:0x0](ffff9a789dfcaa08),name=setup-nfs.sh 00000002:00010000:0.0:1787827241.436004:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4b:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.436011:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.436021:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.436029:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.436034:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4b:0x0](ffff9a789dfcaa08) 00000080:00010000:0.0:1787827241.436037:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4b:0x0](ffff9a789dfcaa08) for lock 0xebdcd48a4298eec9 00000080:00010000:0.0:1787827241.436041:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eec9 from it ffffaa5487e7bcd8 00010000:00010000:0.0:1787827241.436045:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.436054:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.436067:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4b:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.436074:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4b:0x0] 00000008:00010000:0.0:1787827241.436083:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.436086:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f830, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.436115:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.436125:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.436164:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.436197:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476643584:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.437046:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.437068:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.437082:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.437094:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.437107:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.437117:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.437130:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.437140:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.437149:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.437159:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.437181:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476643584:192.168.203.140@tcp:101:cp.0 00000080:00200000:0.0:1787827241.437183:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4b:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1787827241.437189:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.437193:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.437196:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f830 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.437199:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.437201:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f830 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.437209:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4b:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:0.0:1787827241.437230:0:48385:0:(file.c:2652:do_file_write_iter()) START file setup-nfs.sh:[0x240001b71:0x4b:0x0], ppos: 0, count: 1611 00000080:00200000:0.0:1787827241.437238:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 1611 bytes 00000080:00200000:0.0:1787827241.437241:0:48385:0:(file.c:2628:ll_do_tiny_write()) result: 0, original count 1611 00000080:00200000:0.0:1787827241.437244:0:48385:0:(file.c:2090:ll_file_io_generic()) setup-nfs.sh: write ppos: 0, bytes: 1611 00000080:00200000:0.0:1787827241.437247:0:48385:0:(file.c:1848:ll_io_set_mirror()) setup-nfs.sh: desiginated mirror: 0 00000020:00200000:0.0:1787827241.437252:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000113fb5ba[0x110, 1, [0x240001b71:0x4b:0x0] hash] 00000020:00200000:0.0:1787827241.437254:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 1611) 0 0 00000080:00200000:0.0:1787827241.437256:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4b:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827241.437261:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 0] 00020000:00200000:0.0:1787827241.437267:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 1611) 1611, 1611 00020000:00200000:0.0:1787827241.437269:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787827241.437276:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 0] 00000008:00010000:0.0:1787827241.437282:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:0.0:1787827241.437284:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f830, osc lock ffff9a7886bc5380, flags 0x0 00010000:00010000:0.0:1787827241.437291:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419264/166 (0 4095) 00010000:00010000:0.0:1787827241.437307:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.437315:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.437325:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.437337:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533840 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643712:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.437368:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.438206:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787328 00000100:00100000:1.0:1787827241.438210:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787328 00000100:00100000:1.0:1787827241.438240:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 303 00000100:00100000:1.0:1787827241.438247:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838ca580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488787328:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.438255:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.438290:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.438295:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838ca580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488787328:12345-192.168.203.140@tcp:104: Request processed in 50us (136us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.438301:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 303 00010000:00010000:2.0:1787827241.438324:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827241.438331:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.438339:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a05a00 already unused, calling callback (ffffffffc125b410) 00010000:00010000:2.0:1787827241.438343:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.438356:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.438363:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827241.438388:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.438394:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.438414:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.438421:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000e012c5f5/0xebdcd48a4298eed0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcc45 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1787827241.438431:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476643840:192.168.203.140@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787827241.438433:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.438437:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.438440:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827241.438965:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533840 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476643712:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.438979:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.438992:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.439003:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.439279:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787456 00000100:00100000:2.0:1787827241.439288:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787456 00000100:00100000:2.0:1787827241.439322:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 304 00000100:00100000:2.0:1787827241.439332:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c8f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488787456:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.439340:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.439348:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.439354:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.439361:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.439366:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.439372:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.439394:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7885a05200) 00010000:00010000:0.0:1787827241.439411:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.439422:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c8f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75525:x1874671488787456:12345-192.168.203.140@tcp:105: Request processed in 91us (186us total) trans 0 rc 0/0 00010000:00010000:0.0:1787827241.439424:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.439433:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 304 00010000:00010000:0.0:1787827241.439448:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.439468:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,2 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.439479:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.439485:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.439514:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) setup-nfs.sh: write [0, 1611) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.439523:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 1611 bytes 00000080:00200000:0.0:1787827241.439556:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 1611, copied 1611 00000080:00200000:0.0:1787827241.439567:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000017111aa0[3 00000000e5fd515c 2 1 00000000746d8de4 0 0 0] 00000080:00200000:0.0:1787827241.439571:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000fb59cc93 17ffffc0001001 3:0 ffff9a788a7e5cb0 0 lru 00000080:00200000:0.0:1787827241.439580:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000918cddc8 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 00000000c8fc57a7 > 3< 0 0 > 4< 0 0 8 851968 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.439582:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000017111aa0 00000080:00200000:0.0:1787827241.439584:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.439589:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 1, from 0, to 1611 prio 0 00000008:00010000:0.0:1787827241.439607:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000643e3346 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.439621:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcabb8, count 1, dirtied 1 00000080:00200000:0.0:1787827241.439625:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 1 pages 1611 bytes, tot: 1611 00000080:00200000:0.0:1787827241.439627:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) setup-nfs.sh: write bytes 0, result: 1611 00000020:00010000:0.0:1787827241.439638:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.439640:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.439642:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f830 (ffff9a78899db758/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.439646:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.439652:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.439659:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cfb78 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.439661:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f830 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.439666:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 1611 bytes 00000080:00200000:0.0:1787827241.439670:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 0] 00000080:00200000:0.0:1787827241.439674:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4b:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.439677:0:48385:0:(file.c:2240:ll_file_io_generic()) setup-nfs.sh: 2 io complete with rc: 0, result: 1611, restart: 0 00000080:00200000:0.0:1787827241.439681:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4b:0x0] updating i_size 1611 i_blocks 0 00000080:00200000:0.0:1787827241.439684:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 1611 00000080:00200008:0.0:1787827241.439698:0:48385:0:(file.c:479:ll_file_release()) START file setup-nfs.sh:[0x240001b71:0x4b:0x0](ffff9a789dfcaa08), flags 100001 00000002:00100000:0.0:1787827241.439721:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a65321c0 x1874671476643456/t17179869521(17179869521) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.439729:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.439732:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6533840 x1874671476644096/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.439742:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6533840 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644096:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.439771:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.439991:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476643840:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787827241.440492:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6533840 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644096:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.440520:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a65321c0 x1874671476643456/t17179869521(17179869521) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.440559:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.440570:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.440581:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.440592:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.440601:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.440606:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.440608:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.440612:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.440615:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.440620:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.440627:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.440631:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.440635:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.440637:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.440638:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.440640:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=setup-nfs.sh, flags=8 00000080:00200008:0.0:1787827241.440644:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file setup-nfs.sh:[0x240001b71:0x4b:0x0](ffff9a789dfcaa08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.440647:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4b:0x0](ffff9a789dfcaa08),name=setup-nfs.sh 00000002:00010000:0.0:1787827241.440655:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4b:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.440661:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.440667:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.440672:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.440675:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4b:0x0](ffff9a789dfcaa08) 00000080:00010000:0.0:1787827241.440676:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4b:0x0](ffff9a789dfcaa08) for lock 0xebdcd48a4298eec9 00000080:00010000:0.0:1787827241.440678:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eec9 from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.440680:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.440686:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.440695:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4b:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.440698:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4b:0x0] 00000008:00010000:0.0:1787827241.440704:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.440706:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f830, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.440712:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.440719:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.440726:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.440728:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.440735:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a17c6160/0xebdcd48a4298eed7 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xa6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcc4c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.440740:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.440743:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4b:0x0] updating i_size 1611 i_blocks 0 00000020:00010000:0.0:1787827241.440746:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.440747:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.440749:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f830 (ffff9a78899db758/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.440751:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890931638 (ffff9a78899db908/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.440753:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f830 (ffff9a78899db908/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.440758:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4b:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.440785:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.440791:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.440795:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.440801:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.440806:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.440809:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.440811:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.440813:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.440816:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.440825:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.440852:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.440855:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.440859:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.440862:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.440865:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.440872:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sgpdd-survey.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.440876:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sgpdd-survey.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.440883:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sgpdd-survey.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.440903:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.440908:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.440914:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.440923:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644224:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.440952:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.441899:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644224:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.441910:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441918:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441923:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441928:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441933:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441936:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04a00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.441940:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441944:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441948:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441954:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.441957:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.441961:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04a00/0xebdcd48a4298eede lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.441970:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6532580 x1874671476644224/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.441977:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.441987:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.442010:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.442016:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.442021:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.442037:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.442042:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.442045:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.442048:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.442051:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.442053:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.442059:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.442067:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.442071:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.442073:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.442075:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.442077:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.442084:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=sgpdd-survey.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:2.0:1787827241.442087:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sgpdd-survey.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.442092:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sgpdd-survey.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.442109:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.442114:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.442120:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.442129:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644352:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.442158:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.443216:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644352:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.443224:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443230:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443234:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443240:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443245:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443248:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7890906e00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.443250:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443254:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443258:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443261:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.443265:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443269:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906e00/0xebdcd48a4298eee5 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.443277:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a66721c0 x1874671476644352/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.443284:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.443293:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.443343:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.443352:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.443358:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.443365:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.443370:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.443373:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.443375:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.443378:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:2.0:1787827241.443381:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.443386:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.443391:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.443394:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.443397:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.443399:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.443401:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.443407:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.443411:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=sgpdd-survey.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78894b2600, open_flags 80c1, mode 81a4 opened 0 00000080:00200000:2.0:1787827241.443416:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=sgpdd-survey.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:2.0:1787827241.443427:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: sgpdd-survey.sh,[0x240001b71:0x4c:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:2.0:1787827241.443463:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.443472:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.443482:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.443488:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66721c0 x1874671476644608/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.443498:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.443528:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644608:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.443607:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27600 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.445001:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66721c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644608:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.445012:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445025:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445031:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445040:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445047:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445053:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445062:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445068:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.445074:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a66721c0 x1874671476644608/t17179869523(17179869523) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.445083:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a66721c0 x1874671476644608/t17179869523(17179869523) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 656/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.445092:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.445099:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445106:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/76 (0 0) 00000080:00200000:2.0:1787827241.445109:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.445116:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=sgpdd-survey.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:2.0:1787827241.445139:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x4c:0x0](ffff9a789ded21c8) 00000080:00200000:2.0:1787827241.445202:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x4c:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.445206:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x4c:0x0](ffff9a789ded21c8) 00000080:00010000:2.0:1787827241.445210:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4c:0x0](ffff9a789ded21c8) for lock 0xebdcd48a4298eeec 00000080:00010000:2.0:1787827241.445218:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4c:0x0](ffff9a789ded21c8) 00000080:00200008:2.0:1787827241.445238:0:48385:0:(file.c:1090:ll_file_open()) START file sgpdd-survey.sh:[0x240001b71:0x4c:0x0](ffff9a789ded21c8), flags 100301 00000002:00100000:2.0:1787827241.445257:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a66721c0 x1874671476644608/t17179869523(17179869523) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.445293:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eeec from it ffff9a78877da960 00010000:00010000:2.0:1787827241.445298:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445308:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.445326:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4c:0x0](ffff9a789ded21c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787827241.445342:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sgpdd-survey.sh:[0x240001b71:0x4c:0x0](ffff9a789ded21c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.445348:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4c:0x0](ffff9a789ded21c8),name=sgpdd-survey.sh 00000002:00010000:2.0:1787827241.445361:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4c:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.445370:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.445380:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.445389:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.445393:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4c:0x0](ffff9a789ded21c8) 00000080:00010000:2.0:1787827241.445396:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4c:0x0](ffff9a789ded21c8) for lock 0xebdcd48a4298eeec 00000080:00010000:2.0:1787827241.445401:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eeec from it ffffaa5487e7bcd8 00010000:00010000:2.0:1787827241.445404:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.445413:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.445431:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4c:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.445441:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4c:0x0] 00000008:00010000:2.0:1787827241.445457:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.445461:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3ce30, osc lock ffff9a7884062000, flags 0x400001002 00010000:00010000:2.0:1787827241.445493:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.445520:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1787827241.445552:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.445579:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476644736:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.446464:0:10934:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.446478:0:10934:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.446489:0:10934:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.446497:0:10934:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.446522:0:10934:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 5/2,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.446530:0:10934:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.446538:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.446545:0:10934:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.446551:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.446557:0:10934:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.446564:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4c:0x0] updating i_size 0 i_blocks 0 00000100:00100000:1.0:1787827241.446567:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476644736:192.168.203.140@tcp:101:cp.0 00000020:00010000:2.0:1787827241.446570:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890932508 (ffff9a78899db998/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.446574:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890932508 (ffff9a78899db998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.446577:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3ce30 (ffff9a78899db7e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.446580:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890932508 (ffff9a78899db998/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.446582:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3ce30 (ffff9a78899db998/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.446592:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4c:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.446647:0:48385:0:(file.c:2652:do_file_write_iter()) START file sgpdd-survey.sh:[0x240001b71:0x4c:0x0], ppos: 0, count: 1765 00000080:00200000:2.0:1787827241.446658:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 1765 bytes 00000080:00200000:2.0:1787827241.446662:0:48385:0:(file.c:2628:ll_do_tiny_write()) result: 0, original count 1765 00000080:00200000:2.0:1787827241.446666:0:48385:0:(file.c:2090:ll_file_io_generic()) sgpdd-survey.sh: write ppos: 0, bytes: 1765 00000080:00200000:2.0:1787827241.446670:0:48385:0:(file.c:1848:ll_io_set_mirror()) sgpdd-survey.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.446675:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000df4e5d64[0x110, 1, [0x240001b71:0x4c:0x0] hash] 00000020:00200000:2.0:1787827241.446677:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 1765) 0 0 00000080:00200000:2.0:1787827241.446679:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4c:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.446686:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 0] 00020000:00200000:2.0:1787827241.446693:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 1765) 1765, 1765 00020000:00200000:2.0:1787827241.446697:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.446708:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 0] 00000008:00010000:2.0:1787827241.446716:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.446719:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3ce30, osc lock ffff9a7884062000, flags 0x0 00010000:00010000:2.0:1787827241.446725:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/167 (0 4095) 00010000:00010000:2.0:1787827241.446743:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.446750:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.446757:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.446767:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644864:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.446807:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27840 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.447657:0:48371:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787827241.447674:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787584 00000100:00100000:1.0:1787827241.447680:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787584 00000100:00100000:1.0:1787827241.447730:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 305 00000100:00100000:1.0:1787827241.447743:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a643b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488787584:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.447758:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f3ec1bf9/0xebdcd48a4298eef3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcca0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.447826:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.447858:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a643b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77152:x1874671488787584:12345-192.168.203.140@tcp:104: Request processed in 118us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.447873:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 305 00000100:00100000:1.0:1787827241.448083:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838ca940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476644992:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787827241.448478:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476644864:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.448492:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.448514:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.448522:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.448960:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:2.0:1787827241.448999:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787712 00000100:00100000:2.0:1787827241.449005:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787712 00000100:00100000:2.0:1787827241.449035:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 306 00000100:00100000:2.0:1787827241.449041:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6530f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488787712:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.449050:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.449062:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.449073:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.449085:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.449095:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.449106:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.449125:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7886acfa00) 00010000:00010000:0.0:1787827241.449143:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.449155:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6530f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75527:x1874671488787712:12345-192.168.203.140@tcp:105: Request processed in 112us (194us total) trans 0 rc 0/0 00010000:00010000:0.0:1787827241.449162:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.449164:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 306 00010000:00010000:0.0:1787827241.449182:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.449195:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,2 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.449208:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.449219:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.449239:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) sgpdd-survey.sh: write [0, 1765) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.449249:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 1765 bytes 00000080:00200000:0.0:1787827241.449287:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 1765, copied 1765 00000080:00200000:0.0:1787827241.449298:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000720a7750[3 00000000704fc267 2 1 00000000352c32ea 0 0 0] 00000080:00200000:0.0:1787827241.449302:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000008cdb5521 17ffffc0001001 3:0 ffff9a788a7e5ae0 0 lru 00000080:00200000:0.0:1787827241.449311:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000f0c3a1d 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 00000000547aefc4 > 3< 0 0 > 4< 0 0 8 1613824 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.449315:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000720a7750 00000080:00200000:0.0:1787827241.449317:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.449321:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 1, from 0, to 1765 prio 0 00000008:00010000:0.0:1787827241.449353:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000d6e4b99d ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.449373:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded2378, count 1, dirtied 1 00000080:00200000:0.0:1787827241.449378:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 1 pages 1765 bytes, tot: 1765 00000080:00200000:0.0:1787827241.449382:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) sgpdd-survey.sh: write bytes 0, result: 1765 00000020:00010000:0.0:1787827241.449398:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78877cf5d8 (ffff9a78899db998/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.449401:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78877cf5d8 (ffff9a78899db998/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.449405:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3ce30 (ffff9a78899db7e8/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.449410:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.449421:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.449432:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78877cf5d8 (ffff9a78899db998/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.449435:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3ce30 (ffff9a78899db998/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.449441:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 1765 bytes 00000080:00200000:0.0:1787827241.449448:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 0] 00000080:00200000:0.0:1787827241.449454:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4c:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.449458:0:48385:0:(file.c:2240:ll_file_io_generic()) sgpdd-survey.sh: 2 io complete with rc: 0, result: 1765, restart: 0 00000080:00200000:0.0:1787827241.449470:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4c:0x0] updating i_size 1765 i_blocks 0 00000080:00200000:0.0:1787827241.449475:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 1765 00000080:00200008:0.0:1787827241.449497:0:48385:0:(file.c:479:ll_file_release()) START file sgpdd-survey.sh:[0x240001b71:0x4c:0x0](ffff9a789ded21c8), flags 100001 00000002:00100000:0.0:1787827241.449553:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a66721c0 x1874671476644608/t17179869523(17179869523) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.449566:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.449571:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6530000 x1874671476645248/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.449587:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6530000 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645248:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.449625:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.449970:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838ca940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476644992:192.168.203.140@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787827241.450690:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6530000 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645248:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.450707:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a66721c0 x1874671476644608/t17179869523(17179869523) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 584/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.450748:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.450759:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.450769:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.450778:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.450784:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.450787:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.450789:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.450794:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.450796:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.450803:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.450810:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.450814:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.450818:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.450821:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.450825:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.450863:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=sgpdd-survey.sh, flags=8 00000080:00200008:0.0:1787827241.450872:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file sgpdd-survey.sh:[0x240001b71:0x4c:0x0](ffff9a789ded21c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.450880:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4c:0x0](ffff9a789ded21c8),name=sgpdd-survey.sh 00000002:00010000:0.0:1787827241.450892:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4c:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.450901:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.450912:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.450920:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.450925:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4c:0x0](ffff9a789ded21c8) 00000080:00010000:0.0:1787827241.450928:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4c:0x0](ffff9a789ded21c8) for lock 0xebdcd48a4298eeec 00000080:00010000:0.0:1787827241.450932:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298eeec from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.450936:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.450945:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.450962:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4c:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.450967:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4c:0x0] 00000008:00010000:0.0:1787827241.450977:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.450980:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f830, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.450989:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.450997:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.451009:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.451011:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.451017:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d1c2b5b7/0xebdcd48a4298eefa lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcca7 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.451023:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.451027:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4c:0x0] updating i_size 1765 i_blocks 0 00000020:00010000:0.0:1787827241.451030:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a7890932508 (ffff9a78899db998/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.451032:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890932508 (ffff9a78899db998/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.451034:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f830 (ffff9a78899db7e8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.451036:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890932508 (ffff9a78899db998/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.451038:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f830 (ffff9a78899db998/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.451043:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4c:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.451077:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.451083:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.451087:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.451093:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.451099:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.451101:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.451103:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.451106:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.451109:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.451114:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.451119:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.451121:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.451124:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.451125:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.451127:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.451133:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test-framework.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.451137:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test-framework.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.451144:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test-framework.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.451168:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.451173:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.451180:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.451189:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645376:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.451221:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.452274:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645376:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.452283:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452292:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452297:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452302:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452307:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452311:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.452314:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452319:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452325:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452330:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.452334:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452338:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef01 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.452348:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6532580 x1874671476645376/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.452362:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.452374:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.452397:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.452403:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.452409:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.452416:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.452421:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.452425:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.452428:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.452432:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:2.0:1787827241.452436:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.452445:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.452456:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.452461:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.452466:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.452470:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.452472:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.452478:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test-framework.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:2.0:1787827241.452481:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test-framework.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:2.0:1787827241.452488:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test-framework.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787827241.452520:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452525:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.452531:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.452541:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645504:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.452573:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27840 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.453487:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645504:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.453496:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453516:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453522:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453527:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453532:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453535:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7886acea00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:2.0:1787827241.453537:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453543:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453547:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453551:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827241.453554:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453558:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef08 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.453566:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6672d00 x1874671476645504/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.453574:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787827241.453584:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:2.0:1787827241.453629:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.453635:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.453641:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.453649:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.453654:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.453657:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.453659:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.453663:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:2.0:1787827241.453666:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.453671:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.453678:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.453681:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.453684:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.453685:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.453687:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.453693:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.453697:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=test-framework.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78894b3200, open_flags 80c1, mode 81ed opened 0 00000080:00200000:2.0:1787827241.453702:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test-framework.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:2.0:1787827241.453715:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test-framework.sh,[0x240001b71:0x4d:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:2.0:1787827241.453738:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.453743:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.453749:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.453753:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6672d00 x1874671476645760/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.453760:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.453768:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645760:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.453893:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27840 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.455250:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672d00 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476645760:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.455260:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455275:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455285:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455306:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455319:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455329:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455335:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455340:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.455347:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6672d00 x1874671476645760/t17179869525(17179869525) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.455357:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6672d00 x1874671476645760/t17179869525(17179869525) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 664/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.455367:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.455375:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455383:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/77 (0 0) 00000080:00200000:2.0:1787827241.455386:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.455392:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=test-framework.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:2.0:1787827241.455423:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x4d:0x0](ffff9a789ded3248) 00000080:00200000:2.0:1787827241.455492:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x4d:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.455495:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x4d:0x0](ffff9a789ded3248) 00000080:00010000:2.0:1787827241.455498:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4d:0x0](ffff9a789ded3248) for lock 0xebdcd48a4298ef0f 00000080:00010000:2.0:1787827241.455518:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4d:0x0](ffff9a789ded3248) 00000080:00200008:2.0:1787827241.455527:0:48385:0:(file.c:1090:ll_file_open()) START file test-framework.sh:[0x240001b71:0x4d:0x0](ffff9a789ded3248), flags 100301 00000002:00100000:2.0:1787827241.455543:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6672d00 x1874671476645760/t17179869525(17179869525) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.455574:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef0f from it ffff9a78877da960 00010000:00010000:2.0:1787827241.455577:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455600:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.455619:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4d:0x0](ffff9a789ded3248), inode mode 81ed mask 40 00000080:00200008:2.0:1787827241.455632:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file test-framework.sh:[0x240001b71:0x4d:0x0](ffff9a789ded3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787827241.455636:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4d:0x0](ffff9a789ded3248),name=test-framework.sh 00000002:00010000:2.0:1787827241.455645:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4d:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.455651:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.455658:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.455663:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.455666:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4d:0x0](ffff9a789ded3248) 00000080:00010000:2.0:1787827241.455668:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4d:0x0](ffff9a789ded3248) for lock 0xebdcd48a4298ef0f 00000080:00010000:2.0:1787827241.455671:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef0f from it ffffaa5487e7bcd8 00010000:00010000:2.0:1787827241.455674:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.455679:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.455691:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4d:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1787827241.455699:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4d:0x0] 00000008:00010000:2.0:1787827241.455711:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1787827241.455715:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c6b0, osc lock ffff9a7884063998, flags 0x400001002 00010000:00010000:2.0:1787827241.455739:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.455750:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1787827241.455775:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.455796:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476645888:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.456618:0:10936:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.456631:0:10936:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 6/2,0 mode: --/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.456643:0:10936:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.456653:0:10936:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.456663:0:10936:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.456671:0:10936:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.456681:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.456688:0:10936:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.456694:0:10936:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.456700:0:10936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787827241.456709:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476645888:192.168.203.140@tcp:101:cp.0 00000080:00200000:2.0:1787827241.456710:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4d:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1787827241.456716:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.456719:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.456722:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c6b0 (ffff9a78899dbb48/1) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.456726:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.456728:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c6b0 (ffff9a78899db518/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.456738:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4d:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.456964:0:48385:0:(file.c:2652:do_file_write_iter()) START file test-framework.sh:[0x240001b71:0x4d:0x0], ppos: 0, count: 317548 00000080:00200000:2.0:1787827241.456973:0:48385:0:(file.c:2090:ll_file_io_generic()) test-framework.sh: write ppos: 0, bytes: 317548 00000080:00200000:2.0:1787827241.456977:0:48385:0:(file.c:1848:ll_io_set_mirror()) test-framework.sh: desiginated mirror: 0 00000020:00200000:2.0:1787827241.456983:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000afccc7af[0x110, 1, [0x240001b71:0x4d:0x0] hash] 00000020:00200000:2.0:1787827241.456985:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 317548) 0 0 00000080:00200000:2.0:1787827241.456987:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4d:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.456994:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 77] 00020000:00200000:2.0:1787827241.457002:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 317548) 317548, 317548 00020000:00200000:2.0:1787827241.457005:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.457018:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 77] 00000008:00010000:2.0:1787827241.457025:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.457028:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c6b0, osc lock ffff9a7884063998, flags 0x0 00010000:00010000:2.0:1787827241.457035:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca8800 type 11 mode 2 res 10737419264/167 (0 319487) 00010000:00010000:2.0:1787827241.457052:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.457059:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.457067:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.457079:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646016:192.168.203.140@tcp:101:cp.0 00000100:00100000:2.0:1787827241.457109:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27180 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.457937:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787840 00000100:00100000:1.0:1787827241.457943:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787840 00000100:00100000:1.0:1787827241.457986:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 307 00000100:00100000:1.0:1787827241.457993:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c8b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488787840:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.458003:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.458045:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.458050:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c8b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:77344:x1874671488787840:12345-192.168.203.140@tcp:104: Request processed in 58us (168us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.458057:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 307 00010000:00010000:2.0:1787827241.458072:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827241.458082:0:47597:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.458093:0:47597:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7886acf600 already unused, calling callback (ffffffffc125b410) 00010000:00010000:2.0:1787827241.458100:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.458117:0:47597:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.458140:0:47597:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1787827241.458176:0:47597:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.458184:0:47597:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.458202:0:47597:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.458208:0:47597:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000002af69796/0xebdcd48a4298ef16 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dccfb expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1787827241.458228:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886baf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476646144:192.168.203.140@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787827241.458231:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.458235:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827241.458237:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827241.458792:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646016:192.168.203.140@tcp:101:cp.0 00010000:00010000:2.0:1787827241.458807:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.458818:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.458873:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.459215:0:10929:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.140@tcp (source: 12345-192.168.203.140@tcp) 00000100:00100000:2.0:1787827241.459254:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488787968 00000100:00100000:2.0:1787827241.459260:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488787968 00000100:00100000:2.0:1787827241.459294:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 308 00000100:00100000:2.0:1787827241.459305:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78a6532940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488787968:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.459314:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.459326:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->319487] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.459336:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,1 mode: --/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.459351:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.459358:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.459364:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.459382:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7886ace800) 00000100:00100000:2.0:1787827241.459409:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78a6532940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:77167:x1874671488787968:12345-192.168.203.140@tcp:105: Request processed in 107us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.459418:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 308 00010000:00010000:0.0:1787827241.459429:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.459439:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.459449:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.459457:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,2 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.459468:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,2 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.459473:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.459494:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) test-framework.sh: write [0, 317548) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.459521:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459561:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459573:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f9775b70[3 000000002ade88ff 2 1 000000007e439ef5 0 0 0] 00000080:00200000:0.0:1787827241.459578:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000895b81b9 17ffffc0001001 3:0 ffff9a788a7e5658 0 lru 00000080:00200000:0.0:1787827241.459587:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007799f698 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459590:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f9775b70 00000080:00200000:0.0:1787827241.459593:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.459596:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459623:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459628:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002cb875c9[3 000000002ade88ff 2 1 000000007e439ef5 0 0 1] 00000080:00200000:0.0:1787827241.459630:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002d37d6ff 17ffffc0001001 3:0 ffff9a788a7e59f8 1 lru 00000080:00200000:0.0:1787827241.459635:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000000e5cda3b 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459637:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002cb875c9 00000080:00200000:0.0:1787827241.459637:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:0.0:1787827241.459639:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459660:0:48385:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459663:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ed10b487[3 000000002ade88ff 2 1 000000007e439ef5 0 0 2] 00000080:00200000:0.0:1787827241.459665:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000067e7c0a7 17ffffc0001001 3:0 ffff9a78880cc910 2 lru 00000080:00200000:0.0:1787827241.459669:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000042b7ef76 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459671:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ed10b487 00000080:00200000:0.0:1787827241.459672:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:0.0:1787827241.459673:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459684:0:48385:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459687:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b9d12e18[3 000000002ade88ff 2 1 000000007e439ef5 0 0 3] 00000080:00200000:0.0:1787827241.459688:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000798c3414 17ffffc0001001 3:0 ffff9a78880ccd98 3 lru 00000080:00200000:0.0:1787827241.459692:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004d818a8a 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459694:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b9d12e18 00000080:00200000:0.0:1787827241.459695:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:0.0:1787827241.459696:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459706:0:48385:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459711:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b2e47179[3 000000002ade88ff 2 1 000000007e439ef5 0 0 4] 00000080:00200000:0.0:1787827241.459714:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009128cba6 17ffffc0001001 3:0 ffff9a78880cc3a0 4 lru 00000080:00200000:0.0:1787827241.459720:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000040676781 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459722:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b2e47179 00000080:00200000:0.0:1787827241.459723:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:0.0:1787827241.459726:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459750:0:48385:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459756:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008dbba52e[3 000000002ade88ff 2 1 000000007e439ef5 0 0 5] 00000080:00200000:0.0:1787827241.459759:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005b11c6e9 17ffffc0001021 2:0 ffff9a78880cc0e8 5 lru 00000080:00200000:0.0:1787827241.459764:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000873c15cc 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459767:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008dbba52e 00000080:00200000:0.0:1787827241.459768:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:0.0:1787827241.459769:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459781:0:48385:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459784:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b9c2e7ac[3 000000002ade88ff 2 1 000000007e439ef5 0 0 6] 00000080:00200000:0.0:1787827241.459786:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000b9c84a1 17ffffc0001001 3:0 ffff9a78880cc2b8 6 lru 00000080:00200000:0.0:1787827241.459790:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e4436ad9 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459791:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b9c2e7ac 00000080:00200000:0.0:1787827241.459792:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:0.0:1787827241.459793:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459801:0:48385:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459811:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006f2a94ce[3 000000002ade88ff 2 1 000000007e439ef5 0 0 7] 00000080:00200000:0.0:1787827241.459813:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000006dbb6f74 17ffffc0001001 3:0 ffff9a78880cc1d0 7 lru 00000080:00200000:0.0:1787827241.459817:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003d917860 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459819:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006f2a94ce 00000080:00200000:0.0:1787827241.459819:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:0.0:1787827241.459821:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459862:0:48385:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459869:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002a35e585[3 000000002ade88ff 2 1 000000007e439ef5 0 0 8] 00000080:00200000:0.0:1787827241.459872:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000010349326 17ffffc0001001 3:0 ffff9a78880cc740 8 lru 00000080:00200000:0.0:1787827241.459878:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000077f37ef3 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459881:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002a35e585 00000080:00200000:0.0:1787827241.459882:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:0.0:1787827241.459884:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459902:0:48385:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459907:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000636b7ea6[3 000000002ade88ff 2 1 000000007e439ef5 0 0 9] 00000080:00200000:0.0:1787827241.459910:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000892158a8 17ffffc0001001 3:0 ffff9a78880cce80 9 lru 00000080:00200000:0.0:1787827241.459916:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000769c9fa0 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459919:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000636b7ea6 00000080:00200000:0.0:1787827241.459921:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:0.0:1787827241.459923:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459945:0:48385:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459951:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ed9ed0f1[3 000000002ade88ff 2 1 000000007e439ef5 0 0 10] 00000080:00200000:0.0:1787827241.459954:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000058db5c6f 17ffffc0001001 3:0 ffff9a78880cc488 10 lru 00000080:00200000:0.0:1787827241.459958:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008166d8d8 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459959:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ed9ed0f1 00000080:00200000:0.0:1787827241.459960:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:0.0:1787827241.459962:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459973:0:48385:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459975:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008d29652c[3 000000002ade88ff 2 1 000000007e439ef5 0 0 11] 00000080:00200000:0.0:1787827241.459977:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b285d91e 17ffffc0001001 3:0 ffff9a78880ccbc8 11 lru 00000080:00200000:0.0:1787827241.459981:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001c3953a4 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.459983:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008d29652c 00000080:00200000:0.0:1787827241.459984:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:0.0:1787827241.459985:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.459994:0:48385:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:0.0:1787827241.459997:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004e381589[3 000000002ade88ff 2 1 000000007e439ef5 0 0 12] 00000080:00200000:0.0:1787827241.459999:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000076aff0b3 17ffffc0001001 3:0 ffff9a78880cc570 12 lru 00000080:00200000:0.0:1787827241.460002:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f1dddd00 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460004:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004e381589 00000080:00200000:0.0:1787827241.460005:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:0.0:1787827241.460006:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460015:0:48385:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460018:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b1b9d43c[3 000000002ade88ff 2 1 000000007e439ef5 0 0 13] 00000080:00200000:0.0:1787827241.460020:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005986b0fe 17ffffc0001001 3:0 ffff9a78880cc828 13 lru 00000080:00200000:0.0:1787827241.460024:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000036daffe 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460026:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b1b9d43c 00000080:00200000:0.0:1787827241.460027:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:0.0:1787827241.460029:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460039:0:48385:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460042:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a0de4f9a[3 000000002ade88ff 2 1 000000007e439ef5 0 0 14] 00000080:00200000:0.0:1787827241.460044:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000052aef851 17ffffc0001001 3:0 ffff9a78880cc000 14 lru 00000080:00200000:0.0:1787827241.460048:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a71af9e2 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460051:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a0de4f9a 00000080:00200000:0.0:1787827241.460052:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:0.0:1787827241.460054:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460065:0:48385:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460068:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009683bab9[3 000000002ade88ff 2 1 000000007e439ef5 0 0 15] 00000080:00200000:0.0:1787827241.460071:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000cf36b826 17ffffc0001001 3:0 ffff9a78880cccb0 15 lru 00000080:00200000:0.0:1787827241.460076:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000038d14d12 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460079:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009683bab9 00000080:00200000:0.0:1787827241.460081:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:0.0:1787827241.460084:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460099:0:48385:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460104:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000033d7f051[3 000000002ade88ff 2 1 000000007e439ef5 0 0 16] 00000080:00200000:0.0:1787827241.460107:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b447919c 17ffffc0001001 3:0 ffff9a78880ccae0 16 lru 00000080:00200000:0.0:1787827241.460113:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008d414a73 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460116:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000033d7f051 00000080:00200000:0.0:1787827241.460117:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000100:00100000:2.0:1787827241.460118:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886baf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476646144:192.168.203.140@tcp:103:ldlm_bl.0 00000080:00200000:0.0:1787827241.460119:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460153:0:48385:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460157:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004f60af66[3 000000002ade88ff 2 1 000000007e439ef5 0 0 17] 00000080:00200000:0.0:1787827241.460159:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000021e3516a 17ffffc0001001 3:0 ffff9a78880cc658 17 lru 00000080:00200000:0.0:1787827241.460164:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f9328e81 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460165:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004f60af66 00000080:00200000:0.0:1787827241.460166:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:0.0:1787827241.460168:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460180:0:48385:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460183:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007ec49332[3 000000002ade88ff 2 1 000000007e439ef5 0 0 18] 00000080:00200000:0.0:1787827241.460185:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009b11dd8b 17ffffc0001001 3:0 ffff9a78880cc9f8 18 lru 00000080:00200000:0.0:1787827241.460189:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009449bf76 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460190:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007ec49332 00000080:00200000:0.0:1787827241.460191:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:0.0:1787827241.460192:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460208:0:48385:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460211:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ebf758b9[3 000000002ade88ff 2 1 000000007e439ef5 0 0 19] 00000080:00200000:0.0:1787827241.460213:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ae995559 17ffffc0001001 3:0 ffff9a788996f488 19 lru 00000080:00200000:0.0:1787827241.460222:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fc551cdb 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460225:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ebf758b9 00000080:00200000:0.0:1787827241.460225:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:0.0:1787827241.460227:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460240:0:48385:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460243:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000753af124[3 000000002ade88ff 2 1 000000007e439ef5 0 0 20] 00000080:00200000:0.0:1787827241.460245:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000044f5354a 17ffffc0001021 2:0 ffff9a788996fbc8 20 lru 00000080:00200000:0.0:1787827241.460249:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000037ce19f8 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460250:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000753af124 00000080:00200000:0.0:1787827241.460251:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:0.0:1787827241.460252:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460261:0:48385:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460264:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007c98b006[3 000000002ade88ff 2 1 000000007e439ef5 0 0 21] 00000080:00200000:0.0:1787827241.460265:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003f3f037f 17ffffc0001001 3:0 ffff9a788996f570 21 lru 00000080:00200000:0.0:1787827241.460269:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e5920378 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460271:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007c98b006 00000080:00200000:0.0:1787827241.460272:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:0.0:1787827241.460273:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460281:0:48385:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460284:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c45a8023[3 000000002ade88ff 2 1 000000007e439ef5 0 0 22] 00000080:00200000:0.0:1787827241.460286:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000a863be2 17ffffc0001001 3:0 ffff9a788996f828 22 lru 00000080:00200000:0.0:1787827241.460289:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004caf3886 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460291:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c45a8023 00000080:00200000:0.0:1787827241.460293:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:0.0:1787827241.460294:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460308:0:48385:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460313:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000049d2059b[3 000000002ade88ff 2 1 000000007e439ef5 0 0 23] 00000080:00200000:0.0:1787827241.460316:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000dc1329d6 17ffffc0001001 3:0 ffff9a788996f000 23 lru 00000080:00200000:0.0:1787827241.460323:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007d98c128 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460326:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000049d2059b 00000080:00200000:0.0:1787827241.460327:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:0.0:1787827241.460329:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460348:0:48385:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460352:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000002ae9ca[3 000000002ade88ff 2 1 000000007e439ef5 0 0 24] 00000080:00200000:0.0:1787827241.460354:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000016c64f14 17ffffc0001001 3:0 ffff9a788996fcb0 24 lru 00000080:00200000:0.0:1787827241.460358:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000014069aa9 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460360:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000002ae9ca 00000080:00200000:0.0:1787827241.460361:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:0.0:1787827241.460362:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460372:0:48385:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460375:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004e2da414[3 000000002ade88ff 2 1 000000007e439ef5 0 0 25] 00000080:00200000:0.0:1787827241.460377:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a746583a 17ffffc0001001 3:0 ffff9a788996fae0 25 lru 00000080:00200000:0.0:1787827241.460380:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000fc0bfd54 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460382:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004e2da414 00000080:00200000:0.0:1787827241.460383:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:0.0:1787827241.460384:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460393:0:48385:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460396:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000065c94eec[3 000000002ade88ff 2 1 000000007e439ef5 0 0 26] 00000080:00200000:0.0:1787827241.460398:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000ee7f497 17ffffc0001001 3:0 ffff9a788996f658 26 lru 00000080:00200000:0.0:1787827241.460402:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a284b1c4 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460403:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000065c94eec 00000080:00200000:0.0:1787827241.460404:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:0.0:1787827241.460405:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460413:0:48385:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460417:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009f39da50[3 000000002ade88ff 2 1 000000007e439ef5 0 0 27] 00000080:00200000:0.0:1787827241.460420:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b97be5f6 17ffffc0001001 3:0 ffff9a788996f9f8 27 lru 00000080:00200000:0.0:1787827241.460426:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001aa93565 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460428:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009f39da50 00000080:00200000:0.0:1787827241.460429:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:0.0:1787827241.460431:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460447:0:48385:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460452:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009315a743[3 000000002ade88ff 2 1 000000007e439ef5 0 0 28] 00000080:00200000:0.0:1787827241.460456:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000086b07ee8 17ffffc0001001 3:0 ffff9a788996f910 28 lru 00000080:00200000:0.0:1787827241.460462:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ff68e73e 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460464:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009315a743 00000080:00200000:0.0:1787827241.460466:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:0.0:1787827241.460468:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460483:0:48385:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460486:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000008266c62[3 000000002ade88ff 2 1 000000007e439ef5 0 0 29] 00000080:00200000:0.0:1787827241.460488:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000028961040 17ffffc0001001 3:0 ffff9a788996fd98 29 lru 00000080:00200000:0.0:1787827241.460492:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003f98ed19 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460494:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000008266c62 00000080:00200000:0.0:1787827241.460495:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:0.0:1787827241.460496:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460520:0:48385:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460524:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004123faf9[3 000000002ade88ff 2 1 000000007e439ef5 0 0 30] 00000080:00200000:0.0:1787827241.460526:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c8c6d51b 17ffffc0001001 3:0 ffff9a788996f3a0 30 lru 00000080:00200000:0.0:1787827241.460529:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000940f4a9f 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460531:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004123faf9 00000080:00200000:0.0:1787827241.460532:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:0.0:1787827241.460534:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460545:0:48385:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460548:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007de7a6b4[3 000000002ade88ff 2 1 000000007e439ef5 0 0 31] 00000080:00200000:0.0:1787827241.460550:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000028b51e0a 17ffffc0001001 3:0 ffff9a788996f0e8 31 lru 00000080:00200000:0.0:1787827241.460554:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001385e1fd 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460556:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007de7a6b4 00000080:00200000:0.0:1787827241.460557:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:0.0:1787827241.460558:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460566:0:48385:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460569:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000327f8ad4[3 000000002ade88ff 2 1 000000007e439ef5 0 0 32] 00000080:00200000:0.0:1787827241.460570:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000000f7b32eb 17ffffc0001001 3:0 ffff9a788996f2b8 32 lru 00000080:00200000:0.0:1787827241.460574:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007c889a4c 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460576:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000327f8ad4 00000080:00200000:0.0:1787827241.460577:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:0.0:1787827241.460578:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460587:0:48385:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460589:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c971c8f5[3 000000002ade88ff 2 1 000000007e439ef5 0 0 33] 00000080:00200000:0.0:1787827241.460591:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000021d778ba 17ffffc0001001 3:0 ffff9a788996f1d0 33 lru 00000080:00200000:0.0:1787827241.460595:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d25ac478 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460597:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c971c8f5 00000080:00200000:0.0:1787827241.460597:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:0.0:1787827241.460599:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460606:0:48385:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460609:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ed705c94[3 000000002ade88ff 2 1 000000007e439ef5 0 0 34] 00000080:00200000:0.0:1787827241.460611:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000054e33b99 17ffffc0001001 3:0 ffff9a788996f740 34 lru 00000080:00200000:0.0:1787827241.460614:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000928f13f9 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460615:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ed705c94 00000080:00200000:0.0:1787827241.460616:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:0.0:1787827241.460617:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460629:0:48385:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460631:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000093468616[3 000000002ade88ff 2 1 000000007e439ef5 0 0 35] 00000080:00200000:0.0:1787827241.460633:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005f221f8c 17ffffc0001021 2:0 ffff9a788996fe80 35 lru 00000080:00200000:0.0:1787827241.460637:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000085530ebe 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460639:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000093468616 00000080:00200000:0.0:1787827241.460640:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:0.0:1787827241.460641:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460655:0:48385:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460658:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002ed89d57[3 000000002ade88ff 2 1 000000007e439ef5 0 0 36] 00000080:00200000:0.0:1787827241.460660:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c4331aad 17ffffc0001001 3:0 ffff9a788b16ce80 36 lru 00000080:00200000:0.0:1787827241.460664:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004ce25612 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460665:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002ed89d57 00000080:00200000:0.0:1787827241.460666:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:0.0:1787827241.460667:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460676:0:48385:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460679:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c983d523[3 000000002ade88ff 2 1 000000007e439ef5 0 0 37] 00000080:00200000:0.0:1787827241.460681:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000741fe492 17ffffc0001001 3:0 ffff9a788b16c488 37 lru 00000080:00200000:0.0:1787827241.460685:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000007f96f26e 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460686:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c983d523 00000080:00200000:0.0:1787827241.460687:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:0.0:1787827241.460688:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460698:0:48385:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460703:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d7db276c[3 000000002ade88ff 2 1 000000007e439ef5 0 0 38] 00000080:00200000:0.0:1787827241.460705:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004e4ccb9e 17ffffc0001001 3:0 ffff9a788b16cbc8 38 lru 00000080:00200000:0.0:1787827241.460711:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004f22281c 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460713:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d7db276c 00000080:00200000:0.0:1787827241.460714:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:0.0:1787827241.460716:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460731:0:48385:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460736:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000476c15eb[3 000000002ade88ff 2 1 000000007e439ef5 0 0 39] 00000080:00200000:0.0:1787827241.460739:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000025dcec8e 17ffffc0001001 3:0 ffff9a788b16c570 39 lru 00000080:00200000:0.0:1787827241.460746:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ba325f00 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460749:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000476c15eb 00000080:00200000:0.0:1787827241.460750:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:0.0:1787827241.460752:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460765:0:48385:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460768:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d9c71b29[3 000000002ade88ff 2 1 000000007e439ef5 0 0 40] 00000080:00200000:0.0:1787827241.460770:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c53ea1b5 17ffffc0001001 3:0 ffff9a788b16c828 40 lru 00000080:00200000:0.0:1787827241.460774:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000004657308e 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460776:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d9c71b29 00000080:00200000:0.0:1787827241.460777:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:0.0:1787827241.460778:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460787:0:48385:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460790:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e6d88e0f[3 000000002ade88ff 2 1 000000007e439ef5 0 0 41] 00000080:00200000:0.0:1787827241.460791:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b5d7bb1e 17ffffc0001001 3:0 ffff9a788b16c000 41 lru 00000080:00200000:0.0:1787827241.460795:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003a2677e4 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460797:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e6d88e0f 00000080:00200000:0.0:1787827241.460798:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:0.0:1787827241.460799:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460807:0:48385:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460810:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f107265a[3 000000002ade88ff 2 1 000000007e439ef5 0 0 42] 00000080:00200000:0.0:1787827241.460812:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000016656cbf 17ffffc0001001 3:0 ffff9a788b16ccb0 42 lru 00000080:00200000:0.0:1787827241.460816:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000003741b314 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460818:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f107265a 00000080:00200000:0.0:1787827241.460818:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:0.0:1787827241.460819:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460855:0:48385:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460861:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004a101ebf[3 000000002ade88ff 2 1 000000007e439ef5 0 0 43] 00000080:00200000:0.0:1787827241.460864:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000741a06d7 17ffffc0001001 3:0 ffff9a788b16cae0 43 lru 00000080:00200000:0.0:1787827241.460870:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009d5cea4d 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460872:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004a101ebf 00000080:00200000:0.0:1787827241.460874:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:0.0:1787827241.460877:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460895:0:48385:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460900:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006c823224[3 000000002ade88ff 2 1 000000007e439ef5 0 0 44] 00000080:00200000:0.0:1787827241.460903:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000f6e2fbde 17ffffc0001001 3:0 ffff9a788b16c658 44 lru 00000080:00200000:0.0:1787827241.460909:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a68ad3b3 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460911:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006c823224 00000080:00200000:0.0:1787827241.460913:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:0.0:1787827241.460915:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460935:0:48385:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460940:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ccc1058b[3 000000002ade88ff 2 1 000000007e439ef5 0 0 45] 00000080:00200000:0.0:1787827241.460943:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000257ebefd 17ffffc0001001 3:0 ffff9a788b16c9f8 45 lru 00000080:00200000:0.0:1787827241.460949:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000005bf50682 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460952:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ccc1058b 00000080:00200000:0.0:1787827241.460953:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:0.0:1787827241.460955:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460973:0:48385:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:0.0:1787827241.460977:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f594c3f7[3 000000002ade88ff 2 1 000000007e439ef5 0 0 46] 00000080:00200000:0.0:1787827241.460979:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000053bb5a29 17ffffc0001001 3:0 ffff9a788b16c910 46 lru 00000080:00200000:0.0:1787827241.460984:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000008d67d141 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.460985:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f594c3f7 00000080:00200000:0.0:1787827241.460986:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:0.0:1787827241.460987:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.460997:0:48385:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461000:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000023c6e23f[3 000000002ade88ff 2 1 000000007e439ef5 0 0 47] 00000080:00200000:0.0:1787827241.461002:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000061b84eac 17ffffc0001001 3:0 ffff9a788b16cd98 47 lru 00000080:00200000:0.0:1787827241.461006:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d2e67fd3 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461008:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000023c6e23f 00000080:00200000:0.0:1787827241.461008:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:0.0:1787827241.461009:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461018:0:48385:0:(rw26.c:875:ll_write_end()) pos 196608, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461020:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000b7a9cb31[3 000000002ade88ff 2 1 000000007e439ef5 0 0 48] 00000080:00200000:0.0:1787827241.461022:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000096ab0691 17ffffc0001001 3:0 ffff9a788b16c3a0 48 lru 00000080:00200000:0.0:1787827241.461025:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009b26fd29 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461027:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000b7a9cb31 00000080:00200000:0.0:1787827241.461028:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:0.0:1787827241.461029:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 49 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461037:0:48385:0:(rw26.c:875:ll_write_end()) pos 200704, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461040:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000d145de08[3 000000002ade88ff 2 1 000000007e439ef5 0 0 49] 00000080:00200000:0.0:1787827241.461042:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003f5f882b 17ffffc0001001 3:0 ffff9a788b16c0e8 49 lru 00000080:00200000:0.0:1787827241.461046:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000069ca7bb8 49: 1< 0 - - > 2< 200704 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461047:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000d145de08 00000080:00200000:0.0:1787827241.461048:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 50. 00000080:00200000:0.0:1787827241.461049:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 50 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461059:0:48385:0:(rw26.c:875:ll_write_end()) pos 204800, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461062:0:48385:0:(rw26.c:933:ll_write_end()) page@000000004ee73168[3 000000002ade88ff 2 1 000000007e439ef5 0 0 50] 00000080:00200000:0.0:1787827241.461064:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003cc72f92 17ffffc0001021 2:0 ffff9a788b16c2b8 50 lru 00000080:00200000:0.0:1787827241.461067:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e5c6b4e1 50: 1< 0 - - > 2< 204800 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461070:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000004ee73168 00000080:00200000:0.0:1787827241.461071:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 51. 00000080:00200000:0.0:1787827241.461072:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 51 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461080:0:48385:0:(rw26.c:875:ll_write_end()) pos 208896, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461083:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cc0c2024[3 000000002ade88ff 2 1 000000007e439ef5 0 0 51] 00000080:00200000:0.0:1787827241.461085:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d2d9f33b 17ffffc0001001 3:0 ffff9a788b16c1d0 51 lru 00000080:00200000:0.0:1787827241.461089:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c6384256 51: 1< 0 - - > 2< 208896 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461090:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cc0c2024 00000080:00200000:0.0:1787827241.461091:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 52. 00000080:00200000:0.0:1787827241.461092:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 52 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461102:0:48385:0:(rw26.c:875:ll_write_end()) pos 212992, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461104:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f07750f3[3 000000002ade88ff 2 1 000000007e439ef5 0 0 52] 00000080:00200000:0.0:1787827241.461106:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009c64e34f 17ffffc0001001 3:0 ffff9a788b16c740 52 lru 00000080:00200000:0.0:1787827241.461109:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f9d977fe 52: 1< 0 - - > 2< 212992 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461111:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f07750f3 00000080:00200000:0.0:1787827241.461111:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 53. 00000080:00200000:0.0:1787827241.461113:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 53 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461130:0:48385:0:(rw26.c:875:ll_write_end()) pos 217088, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461133:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000abed89ea[3 000000002ade88ff 2 1 000000007e439ef5 0 0 53] 00000080:00200000:0.0:1787827241.461135:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000005df5b43a 17ffffc0001001 3:0 ffff9a78879fb910 53 lru 00000080:00200000:0.0:1787827241.461138:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000053eca3a4 53: 1< 0 - - > 2< 217088 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461140:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000abed89ea 00000080:00200000:0.0:1787827241.461141:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 54. 00000080:00200000:0.0:1787827241.461142:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 54 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461150:0:48385:0:(rw26.c:875:ll_write_end()) pos 221184, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461153:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005702fb7d[3 000000002ade88ff 2 1 000000007e439ef5 0 0 54] 00000080:00200000:0.0:1787827241.461155:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000913783d2 17ffffc0001001 3:0 ffff9a78879fbd98 54 lru 00000080:00200000:0.0:1787827241.461159:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000096d33a4e 54: 1< 0 - - > 2< 221184 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461160:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005702fb7d 00000080:00200000:0.0:1787827241.461161:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 55. 00000080:00200000:0.0:1787827241.461162:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 55 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461171:0:48385:0:(rw26.c:875:ll_write_end()) pos 225280, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461174:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000030cc013e[3 000000002ade88ff 2 1 000000007e439ef5 0 0 55] 00000080:00200000:0.0:1787827241.461176:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c7149d53 17ffffc0001001 3:0 ffff9a78879fb3a0 55 lru 00000080:00200000:0.0:1787827241.461180:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002be6be1f 55: 1< 0 - - > 2< 225280 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461182:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000030cc013e 00000080:00200000:0.0:1787827241.461182:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 56. 00000080:00200000:0.0:1787827241.461183:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 56 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461192:0:48385:0:(rw26.c:875:ll_write_end()) pos 229376, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461194:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000ebe3f5a9[3 000000002ade88ff 2 1 000000007e439ef5 0 0 56] 00000080:00200000:0.0:1787827241.461196:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000026560e51 17ffffc0001001 3:0 ffff9a78879fb0e8 56 lru 00000080:00200000:0.0:1787827241.461199:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000f8b0b902 56: 1< 0 - - > 2< 229376 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461201:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000ebe3f5a9 00000080:00200000:0.0:1787827241.461201:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 57. 00000080:00200000:0.0:1787827241.461202:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 57 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461211:0:48385:0:(rw26.c:875:ll_write_end()) pos 233472, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461213:0:48385:0:(rw26.c:933:ll_write_end()) page@000000009ba1ebfe[3 000000002ade88ff 2 1 000000007e439ef5 0 0 57] 00000080:00200000:0.0:1787827241.461215:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001d820657 17ffffc0001001 3:0 ffff9a78879fb2b8 57 lru 00000080:00200000:0.0:1787827241.461219:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c26963c9 57: 1< 0 - - > 2< 233472 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461220:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000009ba1ebfe 00000080:00200000:0.0:1787827241.461221:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 58. 00000080:00200000:0.0:1787827241.461222:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 58 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461230:0:48385:0:(rw26.c:875:ll_write_end()) pos 237568, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461232:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c2c62d87[3 000000002ade88ff 2 1 000000007e439ef5 0 0 58] 00000080:00200000:0.0:1787827241.461234:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ed1b0ceb 17ffffc0001001 3:0 ffff9a78879fb1d0 58 lru 00000080:00200000:0.0:1787827241.461238:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000039c3c07f 58: 1< 0 - - > 2< 237568 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461239:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c2c62d87 00000080:00200000:0.0:1787827241.461240:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 59. 00000080:00200000:0.0:1787827241.461241:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 59 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461251:0:48385:0:(rw26.c:875:ll_write_end()) pos 241664, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461253:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000066381c45[3 000000002ade88ff 2 1 000000007e439ef5 0 0 59] 00000080:00200000:0.0:1787827241.461255:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000ff7bbc17 17ffffc0001001 3:0 ffff9a78879fb740 59 lru 00000080:00200000:0.0:1787827241.461259:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000a6bc52b4 59: 1< 0 - - > 2< 241664 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461261:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000066381c45 00000080:00200000:0.0:1787827241.461262:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 60. 00000080:00200000:0.0:1787827241.461263:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 60 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461272:0:48385:0:(rw26.c:875:ll_write_end()) pos 245760, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461275:0:48385:0:(rw26.c:933:ll_write_end()) page@000000006146e307[3 000000002ade88ff 2 1 000000007e439ef5 0 0 60] 00000080:00200000:0.0:1787827241.461277:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001fdf3962 17ffffc0001001 3:0 ffff9a78879fbe80 60 lru 00000080:00200000:0.0:1787827241.461280:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000596b390c 60: 1< 0 - - > 2< 245760 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461282:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000006146e307 00000080:00200000:0.0:1787827241.461283:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 61. 00000080:00200000:0.0:1787827241.461284:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 61 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461293:0:48385:0:(rw26.c:875:ll_write_end()) pos 249856, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461297:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c8edf01e[3 000000002ade88ff 2 1 000000007e439ef5 0 0 61] 00000080:00200000:0.0:1787827241.461300:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007386170c 17ffffc0001001 3:0 ffff9a78879fb488 61 lru 00000080:00200000:0.0:1787827241.461305:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000081d25c92 61: 1< 0 - - > 2< 249856 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461307:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c8edf01e 00000080:00200000:0.0:1787827241.461308:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 62. 00000080:00200000:0.0:1787827241.461310:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 62 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461325:0:48385:0:(rw26.c:875:ll_write_end()) pos 253952, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461330:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002704cb35[3 000000002ade88ff 2 1 000000007e439ef5 0 0 62] 00000080:00200000:0.0:1787827241.461334:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b0494af9 17ffffc0001001 3:0 ffff9a78879fbbc8 62 lru 00000080:00200000:0.0:1787827241.461340:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000c4e1924f 62: 1< 0 - - > 2< 253952 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461343:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002704cb35 00000080:00200000:0.0:1787827241.461345:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 63. 00000080:00200000:0.0:1787827241.461347:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 63 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461362:0:48385:0:(rw26.c:875:ll_write_end()) pos 258048, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461365:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e63f7bdb[3 000000002ade88ff 2 1 000000007e439ef5 0 0 63] 00000080:00200000:0.0:1787827241.461367:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000007921935a 17ffffc0001001 3:0 ffff9a78879fb570 63 lru 00000080:00200000:0.0:1787827241.461371:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000dc1ecd4f 63: 1< 0 - - > 2< 258048 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461373:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e63f7bdb 00000080:00200000:0.0:1787827241.461374:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 64. 00000080:00200000:0.0:1787827241.461375:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 64 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461388:0:48385:0:(rw26.c:875:ll_write_end()) pos 262144, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461391:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000c33ff407[3 000000002ade88ff 2 1 000000007e439ef5 0 0 64] 00000080:00200000:0.0:1787827241.461393:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000068856809 17ffffc0001001 3:0 ffff9a78879fb828 64 lru 00000080:00200000:0.0:1787827241.461397:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000006dfaa02c 64: 1< 0 - - > 2< 262144 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461399:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000c33ff407 00000080:00200000:0.0:1787827241.461399:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 65. 00000080:00200000:0.0:1787827241.461400:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 65 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461413:0:48385:0:(rw26.c:875:ll_write_end()) pos 266240, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461417:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000beda183a[3 000000002ade88ff 2 1 000000007e439ef5 0 0 65] 00000080:00200000:0.0:1787827241.461418:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000a58f325d 17ffffc0001021 2:0 ffff9a78879fb000 65 lru 00000080:00200000:0.0:1787827241.461422:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000089d59352 65: 1< 0 - - > 2< 266240 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461423:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000beda183a 00000080:00200000:0.0:1787827241.461424:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 66. 00000080:00200000:0.0:1787827241.461425:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 66 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461435:0:48385:0:(rw26.c:875:ll_write_end()) pos 270336, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461438:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000045d594a3[3 000000002ade88ff 2 1 000000007e439ef5 0 0 66] 00000080:00200000:0.0:1787827241.461440:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b2e10a37 17ffffc0001001 3:0 ffff9a78879fbcb0 66 lru 00000080:00200000:0.0:1787827241.461444:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000001fabfb0b 66: 1< 0 - - > 2< 270336 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461445:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000045d594a3 00000080:00200000:0.0:1787827241.461446:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 67. 00000080:00200000:0.0:1787827241.461447:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 67 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461456:0:48385:0:(rw26.c:875:ll_write_end()) pos 274432, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461458:0:48385:0:(rw26.c:933:ll_write_end()) page@000000008c32d450[3 000000002ade88ff 2 1 000000007e439ef5 0 0 67] 00000080:00200000:0.0:1787827241.461460:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000003c6f4936 17ffffc0001001 3:0 ffff9a78879fbae0 67 lru 00000080:00200000:0.0:1787827241.461463:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000022f25f33 67: 1< 0 - - > 2< 274432 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461465:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000008c32d450 00000080:00200000:0.0:1787827241.461466:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 68. 00000080:00200000:0.0:1787827241.461467:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 68 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461475:0:48385:0:(rw26.c:875:ll_write_end()) pos 278528, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461478:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e45972c6[3 000000002ade88ff 2 1 000000007e439ef5 0 0 68] 00000080:00200000:0.0:1787827241.461480:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000002ac28f24 17ffffc0001001 3:0 ffff9a78879fb658 68 lru 00000080:00200000:0.0:1787827241.461484:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000d3ae6e66 68: 1< 0 - - > 2< 278528 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461486:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e45972c6 00000080:00200000:0.0:1787827241.461487:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 69. 00000080:00200000:0.0:1787827241.461488:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 69 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461497:0:48385:0:(rw26.c:875:ll_write_end()) pos 282624, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461500:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000cec23c53[3 000000002ade88ff 2 1 000000007e439ef5 0 0 69] 00000080:00200000:0.0:1787827241.461517:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000063be6f1a 17ffffc0001001 3:0 ffff9a78879fb9f8 69 lru 00000080:00200000:0.0:1787827241.461521:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000014217422 69: 1< 0 - - > 2< 282624 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461523:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000cec23c53 00000080:00200000:0.0:1787827241.461524:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 70. 00000080:00200000:0.0:1787827241.461526:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 70 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461544:0:48385:0:(rw26.c:875:ll_write_end()) pos 286720, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461547:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f27c750e[3 000000002ade88ff 2 1 000000007e439ef5 0 0 70] 00000080:00200000:0.0:1787827241.461549:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000d112f436 17ffffc0001001 3:0 ffff9a7890a731d0 70 lru 00000080:00200000:0.0:1787827241.461553:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ede455fc 70: 1< 0 - - > 2< 286720 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461554:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f27c750e 00000080:00200000:0.0:1787827241.461555:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 71. 00000080:00200000:0.0:1787827241.461556:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 71 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461565:0:48385:0:(rw26.c:875:ll_write_end()) pos 290816, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461568:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000f720eb8c[3 000000002ade88ff 2 1 000000007e439ef5 0 0 71] 00000080:00200000:0.0:1787827241.461570:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000001cc7c353 17ffffc0001001 3:0 ffff9a7890a73740 71 lru 00000080:00200000:0.0:1787827241.461573:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000af017fa8 71: 1< 0 - - > 2< 290816 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461575:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000f720eb8c 00000080:00200000:0.0:1787827241.461576:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 72. 00000080:00200000:0.0:1787827241.461577:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 72 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461586:0:48385:0:(rw26.c:875:ll_write_end()) pos 294912, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461589:0:48385:0:(rw26.c:933:ll_write_end()) page@000000002b2db3c6[3 000000002ade88ff 2 1 000000007e439ef5 0 0 72] 00000080:00200000:0.0:1787827241.461590:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000118cd85b 17ffffc0001001 3:0 ffff9a7890a73e80 72 lru 00000080:00200000:0.0:1787827241.461594:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000031142d21 72: 1< 0 - - > 2< 294912 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461596:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000002b2db3c6 00000080:00200000:0.0:1787827241.461596:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 73. 00000080:00200000:0.0:1787827241.461597:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 73 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461607:0:48385:0:(rw26.c:875:ll_write_end()) pos 299008, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461610:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000066b02c29[3 000000002ade88ff 2 1 000000007e439ef5 0 0 73] 00000080:00200000:0.0:1787827241.461612:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @0000000020288603 17ffffc0001001 3:0 ffff9a7890a73488 73 lru 00000080:00200000:0.0:1787827241.461616:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ba419369 73: 1< 0 - - > 2< 299008 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461618:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000066b02c29 00000080:00200000:0.0:1787827241.461618:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 74. 00000080:00200000:0.0:1787827241.461619:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 74 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461629:0:48385:0:(rw26.c:875:ll_write_end()) pos 303104, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461632:0:48385:0:(rw26.c:933:ll_write_end()) page@000000005c3bbff8[3 000000002ade88ff 2 1 000000007e439ef5 0 0 74] 00000080:00200000:0.0:1787827241.461634:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000863718a1 17ffffc0001001 3:0 ffff9a7890a73bc8 74 lru 00000080:00200000:0.0:1787827241.461637:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000e71ebcd7 74: 1< 0 - - > 2< 303104 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461639:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000005c3bbff8 00000080:00200000:0.0:1787827241.461639:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 75. 00000080:00200000:0.0:1787827241.461641:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 75 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461649:0:48385:0:(rw26.c:875:ll_write_end()) pos 307200, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461651:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e662915b[3 000000002ade88ff 2 1 000000007e439ef5 0 0 75] 00000080:00200000:0.0:1787827241.461653:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000009ec59641 17ffffc0001001 3:0 ffff9a7890a73570 75 lru 00000080:00200000:0.0:1787827241.461657:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000042fa231c 75: 1< 0 - - > 2< 307200 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461658:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e662915b 00000080:00200000:0.0:1787827241.461658:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 76. 00000080:00200000:0.0:1787827241.461659:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 76 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.461668:0:48385:0:(rw26.c:875:ll_write_end()) pos 311296, len 4096, copied 4096 00000080:00200000:0.0:1787827241.461671:0:48385:0:(rw26.c:933:ll_write_end()) page@0000000069cf0368[3 000000002ade88ff 2 1 000000007e439ef5 0 0 76] 00000080:00200000:0.0:1787827241.461672:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000c9f7eaa4 17ffffc0001001 3:0 ffff9a7890a73828 76 lru 00000080:00200000:0.0:1787827241.461676:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@00000000ea708d71 76: 1< 0 - - > 2< 311296 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461677:0:48385:0:(rw26.c:933:ll_write_end()) end page@0000000069cf0368 00000080:00200000:0.0:1787827241.461677:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 77. 00000080:00200000:0.0:1787827241.461678:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 77 of 0 to 2156 bytes 00000080:00200000:0.0:1787827241.461694:0:48385:0:(rw26.c:875:ll_write_end()) pos 315392, len 2156, copied 2156 00000080:00200000:0.0:1787827241.461697:0:48385:0:(rw26.c:933:ll_write_end()) page@000000007fed2ef2[3 000000002ade88ff 2 1 000000007e439ef5 0 0 77] 00000080:00200000:0.0:1787827241.461700:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @000000004cb28deb 17ffffc0001001 3:0 ffff9a7890a73000 77 lru 00000080:00200000:0.0:1787827241.461706:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@0000000076bffda9 77: 1< 0 - - > 2< 315392 0 0 0x0 0x0 | 0000000023a82262 00000000af27be05 > 3< 0 0 > 4< 0 0 8 823296 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.461709:0:48385:0:(rw26.c:933:ll_write_end()) end page@000000007fed2ef2 00000080:00200000:0.0:1787827241.461710:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 78. 00000080:00200000:0.0:1787827241.461715:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 78, from 0, to 2156 prio 0 00000008:00010000:0.0:1787827241.461745:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461767:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461774:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461779:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461784:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461789:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461794:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461799:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461804:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461810:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461816:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461821:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461825:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461859:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461869:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.461887:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded33f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.461895:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461906:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461922:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461932:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461943:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461953:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461964:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461971:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461976:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461981:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461986:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461992:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.461997:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462002:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462007:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.462015:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded33f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.462019:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462024:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462029:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462034:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462044:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462049:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462054:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462059:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462064:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462069:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462074:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462078:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462083:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462089:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462095:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.462102:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded33f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.462105:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462111:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462120:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462128:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462137:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462146:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462161:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462167:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462173:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462178:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462182:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462187:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462192:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462197:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462202:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.462211:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded33f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.462214:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462220:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462225:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462230:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462235:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462240:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462245:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462250:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462259:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462264:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462269:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462274:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462279:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462283:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462288:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.462299:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded33f8, count 15, dirtied 15 00000008:00010000:0.0:1787827241.462305:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462321:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.462332:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000ba2c8b91 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.462343:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded33f8, count 3, dirtied 3 00000080:00200000:0.0:1787827241.462347:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 78 pages 317548 bytes, tot: 317548 00000080:00200000:0.0:1787827241.462350:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) test-framework.sh: write bytes 0, result: 317548 00000020:00010000:0.0:1787827241.462365:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78879518a8 (ffff9a78899db518/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.462368:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78879518a8 (ffff9a78899db518/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.462370:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c6b0 (ffff9a78899dbb48/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.462374:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.462382:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.462393:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78879518a8 (ffff9a78899db518/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.462395:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c6b0 (ffff9a78899db518/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.462401:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 317548 bytes 00000080:00200000:0.0:1787827241.462408:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 77] 00000080:00200000:0.0:1787827241.462414:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4d:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.462418:0:48385:0:(file.c:2240:ll_file_io_generic()) test-framework.sh: 2 io complete with rc: 0, result: 317548, restart: 0 00000080:00200000:0.0:1787827241.462424:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4d:0x0] updating i_size 317548 i_blocks 0 00000080:00200000:0.0:1787827241.462427:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 317548 00000080:00200008:0.0:1787827241.462455:0:48385:0:(file.c:479:ll_file_release()) START file test-framework.sh:[0x240001b71:0x4d:0x0](ffff9a789ded3248), flags 100001 00000002:00100000:0.0:1787827241.462499:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6672d00 x1874671476645760/t17179869525(17179869525) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.462524:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.462529:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65312c0 x1874671476646400/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.462544:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65312c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646400:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.462591:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.463591:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65312c0 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646400:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.463613:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6672d00 x1874671476645760/t17179869525(17179869525) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.463681:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.463696:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.463714:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.463727:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.463737:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.463744:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.463748:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.463755:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.463759:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.463774:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.463787:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.463794:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.463799:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.463802:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.463805:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.463808:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=test-framework.sh, flags=8 00000080:00200008:0.0:1787827241.463814:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file test-framework.sh:[0x240001b71:0x4d:0x0](ffff9a789ded3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.463818:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4d:0x0](ffff9a789ded3248),name=test-framework.sh 00000002:00010000:0.0:1787827241.463857:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4d:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.463871:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.463883:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.463891:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.463896:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4d:0x0](ffff9a789ded3248) 00000080:00010000:0.0:1787827241.463898:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4d:0x0](ffff9a789ded3248) for lock 0xebdcd48a4298ef0f 00000080:00010000:0.0:1787827241.463901:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef0f from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.463903:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.463909:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.463926:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4d:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.463935:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4d:0x0] 00000008:00010000:0.0:1787827241.463947:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.463949:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2f830, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.463959:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.463967:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.463975:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.463978:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 6/0,1 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.463983:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000040ed45f6/0xebdcd48a4298ef1d lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xa7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd02 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.463988:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.463992:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4d:0x0] updating i_size 317548 i_blocks 0 00000020:00010000:0.0:1787827241.463995:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.463997:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.463999:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2f830 (ffff9a78899dbb48/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.464001:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.464003:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2f830 (ffff9a78899db518/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.464009:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4d:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.464047:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.464053:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.464058:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.464065:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.464069:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.464072:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.464074:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.464077:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.464079:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.464084:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.464089:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.464091:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.464094:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.464095:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.464097:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.464105:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=yaml.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=9 00000080:00200000:0.0:1787827241.464108:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=yaml.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.464115:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: yaml.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.464142:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.464148:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.464161:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.464171:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646528:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.464201:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.465100:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646528:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.465109:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465116:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465121:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465126:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465132:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465136:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a05600 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.465139:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465144:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465149:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465155:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.465161:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465165:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05600/0xebdcd48a4298ef24 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.465175:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6532580 x1874671476646528/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.465184:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.465197:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.465220:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.465226:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.465230:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.465237:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.465242:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.465244:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.465246:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.465248:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.465250:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.465255:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.465262:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.465265:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.465267:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.465269:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.465280:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.465288:0:48385:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=yaml.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=8 00000080:00200000:0.0:1787827241.465291:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=yaml.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=getattr 00000002:00010000:0.0:1787827241.465297:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: yaml.sh,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787827241.465312:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465317:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.465322:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.465330:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646656:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.465358:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.466429:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646656:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.466438:0:48385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466446:0:48385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466451:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466456:0:48385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466461:0:48385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466465:0:48385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a04c00 already unused, calling callback (ffffffffc1b97990) 00010000:00010000:0.0:1787827241.466467:0:48385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466471:0:48385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 4/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466475:0:48385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 5/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466479:0:48385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827241.466482:0:48385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 3/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466486:0:48385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04c00/0xebdcd48a4298ef2b lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.466495:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9a78a6532580 x1874671476646656/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.466518:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787827241.466529:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 30 res 9663683441/1 (0 0) 00000080:00200000:0.0:1787827241.466571:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.466577:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.466583:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.466590:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.466595:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.466598:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.466601:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.466604:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bbf8 00010000:00010000:0.0:1787827241.466606:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.466611:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.466617:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.466620:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:0.0:1787827241.466623:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.466624:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:0.0:1787827241.466626:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.466637:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.466641:0:48385:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=yaml.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), file ffff9a78a6581e00, open_flags 80c1, mode 81a4 opened 0 00000080:00200000:0.0:1787827241.466646:0:48385:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=yaml.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000002:00010000:0.0:1787827241.466659:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: yaml.sh,[0x240001b71:0x4e:0x0]) in obj [0x240001b71:0x1:0x0], intent: open|creat flags 0100302 00010000:00010000:0.0:1787827241.466683:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.466688:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787827241.466696:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787827241.466703:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6532580 x1874671476646912/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787827241.466712:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48385 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.466726:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646912:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.466856:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a788a54b900 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.468382:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6532580 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476646912:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.468397:0:48385:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468415:0:48385:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468424:0:48385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468437:0:48385:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468448:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468456:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468466:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468471:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787827241.468478:0:48385:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a78a6532580 x1874671476646912/t17179869527(17179869527) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787827241.468487:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6532580 x1874671476646912/t17179869527(17179869527) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787827241.468496:0:48385:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787827241.468518:0:48385:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468526:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/78 (0 0) 00000080:00200000:0.0:1787827241.468529:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787827241.468536:0:48385:0:(namei.c:1628:ll_create_it()) VFS Op:name=yaml.sh, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=open|creat 00000080:00200000:0.0:1787827241.468559:0:48385:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x4e:0x0](ffff9a789dfcdb88) 00000080:00200000:0.0:1787827241.468626:0:48385:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x4e:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787827241.468629:0:48385:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x4e:0x0](ffff9a789dfcdb88) 00000080:00010000:0.0:1787827241.468632:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4e:0x0](ffff9a789dfcdb88) for lock 0xebdcd48a4298ef32 00000080:00010000:0.0:1787827241.468639:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4e:0x0](ffff9a789dfcdb88) 00000080:00200008:0.0:1787827241.468648:0:48385:0:(file.c:1090:ll_file_open()) START file yaml.sh:[0x240001b71:0x4e:0x0](ffff9a789dfcdb88), flags 100301 00000002:00100000:0.0:1787827241.468663:0:48385:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a78a6532580 x1874671476646912/t17179869527(17179869527) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787827241.468695:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef32 from it ffff9a7887be0720 00010000:00010000:0.0:1787827241.468700:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468709:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.468727:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4e:0x0](ffff9a789dfcdb88), inode mode 81a4 mask 40 00000080:00200008:0.0:1787827241.468743:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file yaml.sh:[0x240001b71:0x4e:0x0](ffff9a789dfcdb88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.468749:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4e:0x0](ffff9a789dfcdb88),name=yaml.sh 00000002:00010000:0.0:1787827241.468761:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4e:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.468768:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.468775:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.468780:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.468784:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4e:0x0](ffff9a789dfcdb88) 00000080:00010000:0.0:1787827241.468786:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4e:0x0](ffff9a789dfcdb88) for lock 0xebdcd48a4298ef32 00000080:00010000:0.0:1787827241.468789:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef32 from it ffffaa5487e7bcd8 00010000:00010000:0.0:1787827241.468791:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.468796:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.468809:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4e:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.468817:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4e:0x0] 00000008:00010000:0.0:1787827241.468827:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.468853:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2feb0, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.468889:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.468897:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 3/1,0 mode: --/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1787827241.468930:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:1.0:1787827241.468966:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476647040:192.168.203.140@tcp:101:cp.0 00010000:00010000:1.0:1787827241.469973:0:10934:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.469995:0:10934:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 6/2,0 mode: --/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.470012:0:10934:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.470023:0:10934:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787827241.470037:0:10934:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 5/2,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787827241.470048:0:10934:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787827241.470061:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 6/2,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.470071:0:10934:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827241.470081:0:10934:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 5/1,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.470086:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4e:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1787827241.470094:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.470100:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.470107:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2feb0 (ffff9a78899dbb48/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.470115:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_fini():72 00010000:00010000:1.0:1787827241.470116:0:10934:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.470121:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2feb0 (ffff9a78899db518/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.470139:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4e:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:1.0:1787827241.470145:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476647040:192.168.203.140@tcp:101:cp.0 00000080:00200008:0.0:1787827241.470182:0:48385:0:(file.c:2652:do_file_write_iter()) START file yaml.sh:[0x240001b71:0x4e:0x0], ppos: 0, count: 4512 00000080:00200000:0.0:1787827241.470193:0:48385:0:(file.c:2090:ll_file_io_generic()) yaml.sh: write ppos: 0, bytes: 4512 00000080:00200000:0.0:1787827241.470200:0:48385:0:(file.c:1848:ll_io_set_mirror()) yaml.sh: desiginated mirror: 0 00000020:00200000:0.0:1787827241.470207:0:48385:0:(cl_io.c:218:cl_io_rw_init()) header@00000000f10ee83a[0x110, 1, [0x240001b71:0x4e:0x0] hash] 00000020:00200000:0.0:1787827241.470210:0:48385:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 4512) 0 0 00000080:00200000:0.0:1787827241.470213:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4e:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827241.470224:0:48385:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 1] 00020000:00200000:0.0:1787827241.470234:0:48385:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 4512) 4512, 4512 00020000:00200000:0.0:1787827241.470238:0:48385:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787827241.470253:0:48385:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 1] 00000008:00010000:0.0:1787827241.470263:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:0.0:1787827241.470267:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2feb0, osc lock ffff9a7886bc5380, flags 0x0 00010000:00010000:0.0:1787827241.470275:0:48385:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a7886ca9c00 type 11 mode 2 res 11811161088/168 (0 8191) 00010000:00010000:0.0:1787827241.470292:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.470300:0:48385:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.470307:0:48385:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827241.470321:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476647168:192.168.203.140@tcp:101:cp.0 00000100:00100000:0.0:1787827241.470364:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78813efe40 going to sleep for 11 seconds 00000100:00100000:1.0:1787827241.471310:0:48371:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488788096 00000100:00100000:1.0:1787827241.471317:0:48371:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488788096 00000100:00100000:1.0:1787827241.471362:0:48371:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 309 00000100:00100000:1.0:1787827241.471370:0:48371:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488788096:12345-192.168.203.140@tcp:104: 00010000:00010000:1.0:1787827241.471380:0:48371:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827241.471424:0:48371:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787827241.471430:0:48371:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:82310:x1874671488788096:12345-192.168.203.140@tcp:104: Request processed in 61us (170us total) trans 0 rc 0/0 00000100:00100000:1.0:1787827241.471438:0:48371:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 309 00010000:00010000:0.0:1787827241.471447:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787827241.471458:0:47598:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.471471:0:47598:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9a7885a05800 already unused, calling callback (ffffffffc125b410) 00010000:00010000:0.0:1787827241.471479:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400000000 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827241.471494:0:47598:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787827241.471512:0:47598:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:0.0:1787827241.471549:0:47598:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.471555:0:47598:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787827241.471571:0:47598:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 1/0,0 mode: --/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.471577:0:47598:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000645afb94/0xebdcd48a4298ef39 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400000020 nid: local remote: 0x4059aa0aef8dcd56 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.471593:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827241.471595:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827241.471598:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827241.472145:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6672940 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476647168:192.168.203.140@tcp:101:cp.0 00010000:00010000:0.0:1787827241.472157:0:48385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.472166:0:48385:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.472173:0:48385:0:(ldlm_request.c:327:ldlm_completion_ast()) ### waiting indefinitely because of NO_TIMEOUT ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.472484:0:47592:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874671488788224 00000100:00100000:2.0:1787827241.472489:0:47592:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874671488788224 00000100:00100000:2.0:1787827241.472539:0:47592:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.140@tcp, seq: 310 00000100:00100000:2.0:1787827241.472549:0:47592:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9a78838cb840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488788224:12345-192.168.203.140@tcp:105: 00010000:00010000:2.0:1787827241.472558:0:47592:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.472568:0:47592:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.472585:0:47592:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,1 mode: --/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.472597:0:47592:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.472606:0:47592:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10080000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.472613:0:47592:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x80000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.472634:0:47592:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9a7885a04a00) 00010000:00010000:0.0:1787827241.472658:0:48385:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.472668:0:48385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x20000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.472670:0:47592:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9a78838cb840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:LOV_OSC_UUID+4:75526:x1874671488788224:12345-192.168.203.140@tcp:105: Request processed in 122us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1787827241.472679:0:47592:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.140@tcp, seq: 310 00010000:00010000:0.0:1787827241.472681:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10000000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.472699:0:48385:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,2 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x10020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.472711:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.472728:0:48385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.472746:0:48385:0:(vvp_io.c:1339:vvp_io_write_start()) yaml.sh: write [0, 4512) append 0 append lockpos 0 00000080:00200000:0.0:1787827241.472759:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:0.0:1787827241.472796:0:48385:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:0.0:1787827241.472808:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000a6270bbc[3 0000000070dde1d1 2 1 000000007e439ef5 0 0 0] 00000080:00200000:0.0:1787827241.472812:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b48bc05b 17ffffc0001001 3:0 ffff9a7890a73cb0 0 lru 00000080:00200000:0.0:1787827241.472821:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000009a939dfb 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 00000000ba8512da > 3< 0 0 > 4< 0 0 8 1585152 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.472824:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000a6270bbc 00000080:00200000:0.0:1787827241.472825:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:0.0:1787827241.472852:0:48385:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 416 bytes 00000080:00200000:0.0:1787827241.472882:0:48385:0:(rw26.c:875:ll_write_end()) pos 4096, len 416, copied 416 00000080:00200000:0.0:1787827241.472888:0:48385:0:(rw26.c:933:ll_write_end()) page@00000000e3b84f3a[3 0000000070dde1d1 2 1 000000007e439ef5 0 0 1] 00000080:00200000:0.0:1787827241.472892:0:48385:0:(rw26.c:933:ll_write_end()) vmpage @00000000b0168f5a 17ffffc0001001 3:0 ffff9a7890a73ae0 1 lru 00000080:00200000:0.0:1787827241.472899:0:48385:0:(rw26.c:933:ll_write_end()) osc-page@000000002040029e 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000043fddcb7 00000000ba8512da > 3< 0 0 > 4< 0 0 8 1585152 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:0.0:1787827241.472902:0:48385:0:(rw26.c:933:ll_write_end()) end page@00000000e3b84f3a 00000080:00200000:0.0:1787827241.472903:0:48385:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:0.0:1787827241.472907:0:48385:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 2, from 0, to 416 prio 0 00000008:00010000:0.0:1787827241.472933:0:48385:0:(osc_cache.c:858:osc_extent_find()) ### extent: 00000000e4e82e5a ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1787827241.472961:0:48385:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 00000000e4e82e5a ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1787827241.472976:0:48385:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789dfcdd38, count 2, dirtied 2 00000080:00200000:0.0:1787827241.472980:0:48385:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 2 pages 4512 bytes, tot: 4512 00000080:00200000:0.0:1787827241.472983:0:48385:0:(vvp_io.c:1421:vvp_io_write_start()) yaml.sh: write bytes 0, result: 4512 00000020:00010000:0.0:1787827241.472998:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78879518a8 (ffff9a78899db518/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.473000:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78879518a8 (ffff9a78899db518/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.473002:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2feb0 (ffff9a78899dbb48/1) at cl_lock_cancel():142 00010000:00010000:0.0:1787827241.473006:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827241.473013:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:0.0:1787827241.473020:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78879518a8 (ffff9a78899db518/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.473021:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2feb0 (ffff9a78899db518/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.473027:0:48385:0:(vvp_io.c:533:vvp_io_advance()) advancing 4512 bytes 00000080:00200000:0.0:1787827241.473032:0:48385:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 1] 00000080:00200000:0.0:1787827241.473037:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4e:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.473040:0:48385:0:(file.c:2240:ll_file_io_generic()) yaml.sh: 2 io complete with rc: 0, result: 4512, restart: 0 00000080:00200000:0.0:1787827241.473044:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4e:0x0] updating i_size 4512 i_blocks 0 00000080:00200000:0.0:1787827241.473047:0:48385:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 4512 00000080:00200008:0.0:1787827241.473067:0:48385:0:(file.c:479:ll_file_release()) START file yaml.sh:[0x240001b71:0x4e:0x0](ffff9a789dfcdb88), flags 100001 00000002:00100000:0.0:1787827241.473102:0:48385:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a78a6532580 x1874671476646912/t17179869527(17179869527) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787827241.473111:0:48385:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787827241.473116:0:48385:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3c780 x1874671476647552/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787827241.473127:0:48385:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476647552:192.168.203.140@tcp:35:cp.0 00000100:00100000:0.0:1787827241.473157:0:48385:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78813efe40 going to sleep for 11 seconds 00000100:00100000:0.0:1787827241.473990:0:48385:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c780 pname:cluuid:pid:xid:nid:opc:job cp:97196825-2cb9-49ba-a47d-1455a997ce3a:48385:1874671476647552:192.168.203.140@tcp:35:cp.0 00000002:00100000:0.0:1787827241.474011:0:48385:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a78a6532580 x1874671476646912/t17179869527(17179869527) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787827241.474057:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.474070:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.474079:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.474088:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.474094:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.474098:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.474100:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.474105:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e7bb58 00010000:00010000:0.0:1787827241.474107:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.474113:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.474121:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.474126:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:0.0:1787827241.474129:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.474132:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:0.0:1787827241.474135:0:48385:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.474137:0:48385:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=yaml.sh, flags=8 00000080:00200008:0.0:1787827241.474142:0:48385:0:(file.c:6313:ll_getattr_dentry()) START file yaml.sh:[0x240001b71:0x4e:0x0](ffff9a789dfcdb88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787827241.474146:0:48385:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x240001b71:0x4e:0x0](ffff9a789dfcdb88),name=yaml.sh 00000002:00010000:0.0:1787827241.474155:0:48385:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x4e:0x0]) in obj [0x240001b71:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.474160:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.474166:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827241.474171:0:48385:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x240001b71:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.474173:0:48385:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4e:0x0](ffff9a789dfcdb88) 00000080:00010000:0.0:1787827241.474175:0:48385:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4e:0x0](ffff9a789dfcdb88) for lock 0xebdcd48a4298ef32 00000080:00010000:0.0:1787827241.474177:0:48385:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef32 from it ffffaa5487e7bcb8 00010000:00010000:0.0:1787827241.474179:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.474183:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827241.474194:0:48385:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4e:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787827241.474199:0:48385:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x240001b71:0x4e:0x0] 00000008:00010000:0.0:1787827241.474206:0:48385:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787827241.474208:0:48385:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7889b2feb0, osc lock ffff9a7886bc5380, flags 0x400001002 00010000:00010000:0.0:1787827241.474215:0:48385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.474222:0:48385:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1787827241.474229:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1787827241.474232:0:48385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 6/0,1 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x810020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827241.474236:0:48385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d043da04/0xebdcd48a4298ef40 lrc: 5/0,0 mode: PW/PW res: [0x2c0000400:0xa8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dcd5d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1787827241.474242:0:48385:0:(cl_io.c:1853:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1787827241.474247:0:48385:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x4e:0x0] updating i_size 4512 i_blocks 0 00000020:00010000:0.0:1787827241.474251:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_release():245 00000020:00010000:0.0:1787827241.474254:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.474257:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7889b2feb0 (ffff9a78899dbb48/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787827241.474261:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78909333d8 (ffff9a78899db518/0) at cl_lock_fini():72 00000020:00010000:0.0:1787827241.474263:0:48385:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7889b2feb0 (ffff9a78899db518/1) at cl_lock_fini():72 00000080:00200000:0.0:1787827241.474270:0:48385:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4e:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787827241.480664:0:48386:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.480692:0:48386:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.480714:0:48386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.480733:0:48386:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.480750:0:48386:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.480759:0:48386:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.480763:0:48386:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.480773:0:48386:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e83e18 00010000:00010000:0.0:1787827241.480777:0:48386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.480785:0:48386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.480803:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 101 00000080:00200000:0.0:1787827241.480816:0:48386:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.480823:0:48386:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.480864:0:48386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.480877:0:48386:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.480886:0:48386:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.480891:0:48386:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.480895:0:48386:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.480899:0:48386:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e83be0 00010000:00010000:0.0:1787827241.480903:0:48386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.480917:0:48386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.480926:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.480932:0:48386:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=1536 00000080:00200000:0.0:1787827241.480942:0:48386:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:0.0:1787827241.480948:0:48386:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787827241.480953:0:48386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.480960:0:48386:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.480968:0:48386:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787827241.480972:0:48386:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:0.0:1787827241.480974:0:48386:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:0.0:1787827241.480978:0:48386:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e83c18 00010000:00010000:0.0:1787827241.480981:0:48386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787827241.480989:0:48386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787827241.480999:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.481004:0:48386:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=3 00000080:00200000:0.0:1787827241.481009:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 101 00000080:00200000:0.0:1787827241.481018:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.481024:0:48386:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=1536 00000080:00200000:0.0:1787827241.481030:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.481032:0:48386:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=3 00000080:00200000:0.0:1787827241.481035:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 101 00000080:00200000:0.0:1787827241.481039:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:0.0:1787827241.481049:0:48386:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d_f6.sanity-scrub, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:0.0:1787827241.481053:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.481058:0:48386:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:0.0:1787827241.481063:0:48386:0:(namei.c:2239:do_mkdir()) VFS Op:name=d_f6.sanity-scrub, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) 00000020:00100000:0.0:1787827241.481142:0:48386:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787827241.481158:0:48386:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7890a3d2c0 x1874671476647808/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 568/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827241.481188:0:48386:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3d2c0 pname:cluuid:pid:xid:nid:opc:job mkdir:97196825-2cb9-49ba-a47d-1455a997ce3a:48386:1874671476647808:192.168.203.140@tcp:36:mkdir.0 00000100:00100000:0.0:1787827241.481280:0:48386:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78813efe40 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.483273:0:48386:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3d2c0 pname:cluuid:pid:xid:nid:opc:job mkdir:97196825-2cb9-49ba-a47d-1455a997ce3a:48386:1874671476647808:192.168.203.140@tcp:36:mkdir.0 00000080:00200000:2.0:1787827241.483326:0:48386:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x4f:0x0](ffff9a789ded6c08) 00000080:00200000:2.0:1787827241.483338:0:48386:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001b71:0x4f:0x0], updating i_size 4096 00000080:00200000:2.0:1787827241.487783:0:48387:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.487810:0:48387:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.487862:0:48387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.487881:0:48387:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.487893:0:48387:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.487901:0:48387:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.487906:0:48387:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.487913:0:48387:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e73bf8 00010000:00010000:2.0:1787827241.487918:0:48387:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.487929:0:48387:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.487947:0:48387:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.487956:0:48387:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.487962:0:48387:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.487966:0:48387:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.487970:0:48387:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.487980:0:48387:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d_f6.sanity-scrub, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=25 00000080:00200000:2.0:1787827241.487985:0:48387:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d_f6.sanity-scrub, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), intent=0 00000002:00010000:2.0:1787827241.488002:0:48387:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d_f6.sanity-scrub,[0x0:0x0:0x0]) in obj [0x240001b71:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1787827241.488053:0:48387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.488063:0:48387:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.488076:0:48387:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.488103:0:48387:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886baf480 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476647936:192.168.203.140@tcp:101:touch.0 00000100:00100000:2.0:1787827241.488192:0:48387:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27180 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.489667:0:48387:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886baf480 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476647936:192.168.203.140@tcp:101:touch.0 00010000:00010000:2.0:1787827241.489683:0:48387:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 4/1,0 mode: --/CR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489701:0:48387:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 4/1,0 mode: --/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489712:0:48387:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 4/1,0 mode: --/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489726:0:48387:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489739:0:48387:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489751:0:48387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 5/2,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489761:0:48387:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 5/2,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489768:0:48387:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.489781:0:48387:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff9a7886baf480 x1874671476647936/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 592/584 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.489797:0:48387:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489807:0:48387:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/79 (0 0) 00000080:00200000:2.0:1787827241.489812:0:48387:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x240001b71:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.489856:0:48387:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001b71:0x4f:0x0], updating i_size 4096 00000080:00200000:2.0:1787827241.489861:0:48387:0:(namei.c:147:ll_iget()) got inode: [0x240001b71:0x4f:0x0](ffff9a789ded6c08): rc = 0 00000080:00010000:2.0:1787827241.489868:0:48387:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x4f:0x0](ffff9a789ded6c08) for lock 0xebdcd48a4298ef47 00000080:00010000:2.0:1787827241.489883:0:48387:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x4f:0x0](ffff9a789ded6c08) 00000080:00010000:2.0:1787827241.489887:0:48387:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef47 from it ffffaa5487e73ac8 00010000:00010000:2.0:1787827241.489892:0:48387:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.489903:0:48387:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.489927:0:48387:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.489939:0:48387:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.489944:0:48387:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f1, dir=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), file ffff9a7888f33b00, open_flags 8941, mode 81b6 opened 0 00000080:00200000:2.0:1787827241.489952:0:48387:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f1, dir=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), intent=open|creat 00000002:00010000:2.0:1787827241.489970:0:48387:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f1,[0x240001b71:0x50:0x0]) in obj [0x240001b71:0x4f:0x0], intent: open|creat flags 0104502 00010000:00010000:2.0:1787827241.490003:0:48387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.490012:0:48387:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.490027:0:48387:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.490034:0:48387:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886baf480 x1874671476648192/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.490044:0:48387:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.490058:0:48387:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886baf480 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648192:192.168.203.140@tcp:101:touch.0 00000100:00100000:2.0:1787827241.490165:0:48387:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27180 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.491491:0:48387:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886baf480 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648192:192.168.203.140@tcp:101:touch.0 00010000:00010000:2.0:1787827241.491518:0:48387:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491534:0:48387:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491542:0:48387:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491553:0:48387:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491564:0:48387:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491576:0:48387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491591:0:48387:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491599:0:48387:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.491611:0:48387:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a7886baf480 x1874671476648192/t17179869530(17179869530) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.491627:0:48387:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a7886baf480 x1874671476648192/t17179869530(17179869530) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.491641:0:48387:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.491654:0:48387:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491663:0:48387:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/80 (0 0) 00000080:00200000:2.0:1787827241.491668:0:48387:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x4f:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.491677:0:48387:0:(namei.c:1628:ll_create_it()) VFS Op:name=f1, dir=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), intent=open|creat 00000080:00200000:2.0:1787827241.491701:0:48387:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x50:0x0](ffff9a789ded42c8) 00000080:00200000:2.0:1787827241.491799:0:48387:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x50:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.491806:0:48387:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x50:0x0](ffff9a789ded42c8) 00000080:00010000:2.0:1787827241.491810:0:48387:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x50:0x0](ffff9a789ded42c8) for lock 0xebdcd48a4298ef4e 00000080:00010000:2.0:1787827241.491818:0:48387:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x50:0x0](ffff9a789ded42c8) 00000080:00200008:2.0:1787827241.491827:0:48387:0:(file.c:1090:ll_file_open()) START file f1:[0x240001b71:0x50:0x0](ffff9a789ded42c8), flags 104501 00000002:00100000:2.0:1787827241.491879:0:48387:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a7886baf480 x1874671476648192/t17179869530(17179869530) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.491919:0:48387:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef4e from it ffff9a78877da960 00010000:00010000:2.0:1787827241.491925:0:48387:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.491934:0:48387:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.491953:0:48387:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x50:0x0](ffff9a789ded42c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787827241.491981:0:48387:0:(llite_lib.c:2427:ll_setattr_raw()) START file f1:[0x240001b71:0x50:0x0](ffff9a789ded42c8) current size 0, valid attrs 20070, mode 9a78, uid -2014888368, gid -21932, new size 18446744071879195159, atime 1787827241.000000000, mtime 1787827241.000000000, ctime 1787827241.000000000 00010000:00010000:2.0:1787827241.491999:0:48387:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.492017:0:48387:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x50:0x0]: losing layout lock 00010000:00010000:2.0:1787827241.492046:0:48387:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.492055:0:48387:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787827241.492058:0:48387:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acea00/0xebdcd48a4298ef4e lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x50:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcdaa expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787827241.492078:0:48387:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.492086:0:48387:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bacf00 x1874671476648448/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.492104:0:48387:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bacf00 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648448:192.168.203.140@tcp:36:touch.0 00000100:00100000:2.0:1787827241.492150:0:48387:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.493078:0:48387:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bacf00 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648448:192.168.203.140@tcp:36:touch.0 00000080:00200000:2.0:1787827241.493106:0:48387:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x50:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00010000:2.0:1787827241.493122:0:48387:0:(file.c:7076:ll_layout_intent()) ### lustre: requeue layout lock for file [0x240001b71:0x50:0x0](ffff9a789ded42c8) 00000002:00010000:2.0:1787827241.493130:0:48387:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001b71:0x50:0x0]) in obj [0x240001b71:0x50:0x0], intent: layout flags 0 00010000:00010000:2.0:1787827241.493169:0:48387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.493179:0:48387:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.493188:0:48387:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: --/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 48387 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.493202:0:48387:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bad680 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648576:192.168.203.140@tcp:101:touch.0 00000100:00100000:2.0:1787827241.493239:0:48387:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.493971:0:48387:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bad680 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648576:192.168.203.140@tcp:101:touch.0 00010000:00010000:2.0:1787827241.493983:0:48387:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 4/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.493991:0:48387:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 4/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.494003:0:48387:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff9a7886bad680 x1874671476648576/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 376/400 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.494014:0:48387:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.494031:0:48387:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.494038:0:48387:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 16 res 9663683441/80 (0 0) 00000080:00200000:2.0:1787827241.494041:0:48387:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x240001b71:0x50:0x0] suppgids 0 0: rc 0 00000080:00010000:2.0:1787827241.494051:0:48387:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x50:0x0](ffff9a789ded42c8) for lock 0xebdcd48a4298ef55 00000080:00010000:2.0:1787827241.494054:0:48387:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef55 from it ffffaa5487e73a78 00010000:00010000:2.0:1787827241.494057:0:48387:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.494063:0:48387:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 2/0,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.494073:0:48387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 2/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.494079:0:48387:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 2/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:2.0:1787827241.494085:0:48387:0:(file.c:6959:ll_layout_lock_set()) ### file [0x240001b71:0x50:0x0](00000000d15c7954) being reconfigured ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.494102:0:48387:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x50:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:2.0:1787827241.494105:0:48387:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 3/1,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787827241.494111:0:48387:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 2/0,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00020000:00200000:2.0:1787827241.494128:0:48387:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1787827241.494227:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bad680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476648704:192.168.203.140@tcp:2:touch.0 00000080:00200000:2.0:1787827241.495083:0:48387:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x50:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787827241.495097:0:48387:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f1:[0x240001b71:0x50:0x0](ffff9a789ded42c8) current size 0, valid attrs 100201f0, mode 9a78, uid -2014888368, gid -21932, new size 18446744071879195159, atime 1787827241.000000000, mtime 1787827241.000000000, ctime 1787827241.000000000, rc 0 00000080:00200008:2.0:1787827241.495115:0:48387:0:(file.c:479:ll_file_release()) START file f1:[0x240001b71:0x50:0x0](ffff9a789ded42c8), flags 104001 00000002:00100000:2.0:1787827241.495151:0:48387:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a7886baf480 x1874671476648192/t17179869530(17179869530) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787827241.495162:0:48387:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787827241.495168:0:48387:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886baed00 x1874671476648960/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787827241.495183:0:48387:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886baed00 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648960:192.168.203.140@tcp:35:touch.0 00000100:00100000:2.0:1787827241.495221:0:48387:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.496103:0:48387:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886baed00 pname:cluuid:pid:xid:nid:opc:job touch:97196825-2cb9-49ba-a47d-1455a997ce3a:48387:1874671476648960:192.168.203.140@tcp:35:touch.0 00000002:00100000:2.0:1787827241.496117:0:48387:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a7886baf480 x1874671476648192/t17179869530(17179869530) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.499416:0:48388:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.499437:0:48388:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.499453:0:48388:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.499468:0:48388:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.499477:0:48388:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.499484:0:48388:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.499488:0:48388:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.499495:0:48388:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa54816c7bf8 00010000:00010000:2.0:1787827241.499499:0:48388:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.499523:0:48388:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.499538:0:48388:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.499546:0:48388:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.499552:0:48388:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.499556:0:48388:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=25 00000080:00200000:2.0:1787827241.499561:0:48388:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.499565:0:48388:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d_f6.sanity-scrub, flags=25 00000080:00200000:2.0:1787827241.499570:0:48388:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.499581:0:48388:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.499586:0:48388:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f2, dir=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), file ffff9a78997be780, open_flags 8241, mode 81b6 opened 0 00000080:00200000:2.0:1787827241.499594:0:48388:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f2, dir=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), intent=open|creat 00000002:00010000:2.0:1787827241.499618:0:48388:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f2,[0x240001b71:0x51:0x0]) in obj [0x240001b71:0x4f:0x0], intent: open|creat flags 0101102 00010000:00010000:2.0:1787827241.499664:0:48388:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 48388 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.499675:0:48388:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48388 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787827241.499684:0:48388:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787827241.499694:0:48388:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bae940 x1874671476649216/t0(0) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787827241.499705:0:48388:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 3/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 48388 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.499720:0:48388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bae940 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48388:1874671476649216:192.168.203.140@tcp:101:dd.0 00000100:00100000:2.0:1787827241.499877:0:48388:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7890f27c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.501457:0:48388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bae940 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48388:1874671476649216:192.168.203.140@tcp:101:dd.0 00010000:00010000:2.0:1787827241.501472:0:48388:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 4/0,1 mode: --/CW res: [0x240001b71:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501486:0:48388:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501493:0:48388:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 4/0,1 mode: --/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501516:0:48388:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501525:0:48388:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 4/0,1 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501534:0:48388:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501540:0:48388:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 5/1,1 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501550:0:48388:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 4/1,0 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787827241.501558:0:48388:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9a7886bae940 x1874671476649216/t17179869533(17179869533) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787827241.501567:0:48388:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a7886bae940 x1874671476649216/t17179869533(17179869533) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 648/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787827241.501577:0:48388:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787827241.501586:0:48388:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501593:0:48388:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9a78841cfc00 type 13 mode 4 res 9663683441/81 (0 0) 00000080:00200000:2.0:1787827241.501597:0:48388:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001b71:0x4f:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787827241.501605:0:48388:0:(namei.c:1628:ll_create_it()) VFS Op:name=f2, dir=[0x240001b71:0x4f:0x0](ffff9a789ded6c08), intent=open|creat 00000080:00200000:2.0:1787827241.501628:0:48388:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x51:0x0](ffff9a789ded7448) 00000080:00200000:2.0:1787827241.501693:0:48388:0:(file.c:6854:ll_layout_conf()) [0x240001b71:0x51:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787827241.501696:0:48388:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001b71:0x51:0x0](ffff9a789ded7448) 00000080:00010000:2.0:1787827241.501699:0:48388:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001b71:0x51:0x0](ffff9a789ded7448) for lock 0xebdcd48a4298ef5c 00000080:00010000:2.0:1787827241.501705:0:48388:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001b71:0x51:0x0](ffff9a789ded7448) 00000080:00200008:2.0:1787827241.501713:0:48388:0:(file.c:1090:ll_file_open()) START file f2:[0x240001b71:0x51:0x0](ffff9a789ded7448), flags 101101 00000002:00100000:2.0:1787827241.501727:0:48388:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9a7886bae940 x1874671476649216/t17179869533(17179869533) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787827241.501757:0:48388:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298ef5c from it ffff9a78877da000 00010000:00010000:2.0:1787827241.501761:0:48388:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 3/1,0 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.501767:0:48388:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787827241.501781:0:48388:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x51:0x0](ffff9a789ded7448), inode mode 81a4 mask 40 00000080:00200008:2.0:1787827241.503260:0:48388:0:(file.c:2652:do_file_write_iter()) START file f2:[0x240001b71:0x51:0x0], ppos: 0, count: 1048576 00000080:00200000:2.0:1787827241.503270:0:48388:0:(file.c:2090:ll_file_io_generic()) f2: write ppos: 0, bytes: 1048576 00000080:00200000:2.0:1787827241.503274:0:48388:0:(file.c:1848:ll_io_set_mirror()) f2: desiginated mirror: 0 00000020:00200000:2.0:1787827241.503281:0:48388:0:(cl_io.c:218:cl_io_rw_init()) header@000000002df2d04f[0x110, 1, [0x240001b71:0x51:0x0] hash] 00000020:00200000:2.0:1787827241.503283:0:48388:0:(cl_io.c:218:cl_io_rw_init()) io range: 2 [0, 1048576) 0 0 00000080:00200000:2.0:1787827241.503288:0:48388:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x51:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787827241.503303:0:48388:0:(file.c:2170:ll_file_io_generic()) Range lock [0, 255] 00020000:00200000:2.0:1787827241.503311:0:48388:0:(lov_io.c:1445:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 1048576) 1048576, 1048576 00020000:00200000:2.0:1787827241.503314:0:48388:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.503332:0:48388:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 255] 00000008:00010000:2.0:1787827241.503350:0:48388:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1787827241.503356:0:48388:0:(osc_lock.c:1320:osc_lock_init()) ### lock ffff9a7890c3c8b0, osc lock ffff9a78840624e0, flags 0x0 00010000:00010000:2.0:1787827241.503397:0:48388:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.503404:0:48388:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.503411:0:48388:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827241.503425:0:48388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886baf0c0 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48388:1874671476649344:192.168.203.140@tcp:101:dd.0 00000100:00100000:2.0:1787827241.503478:0:48388:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.504366:0:48388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886baf0c0 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48388:1874671476649344:192.168.203.140@tcp:101:dd.0 00010000:00010000:2.0:1787827241.504380:0:48388:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.504394:0:48388:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.504405:0:48388:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.504420:0:48388:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.504433:0:48388:0:(osc_lock.c:192:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,2 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.504445:0:48388:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 5/0,2 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.504451:0:48388:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.504468:0:48388:0:(vvp_io.c:1339:vvp_io_write_start()) f2: write [0, 1048576) append 0 append lockpos 0 00000080:00200000:2.0:1787827241.504480:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504524:0:48388:0:(rw26.c:875:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504534:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000020f80f31[3 00000000f7922827 2 1 000000000104970a 0 0 0] 00000080:00200000:2.0:1787827241.504537:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000c3d1a39 17ffffc0001001 3:0 ffff9a78a6521d98 0 lru 00000080:00200000:2.0:1787827241.504544:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000010206b15 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504546:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000020f80f31 00000080:00200000:2.0:1787827241.504547:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 1. 00000080:00200000:2.0:1787827241.504550:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 1 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504567:0:48388:0:(rw26.c:875:ll_write_end()) pos 4096, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504570:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000b50b51e[3 00000000f7922827 2 1 000000000104970a 0 0 1] 00000080:00200000:2.0:1787827241.504573:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000224a9e82 17ffffc0001001 3:0 ffff9a78a65213a0 1 lru 00000080:00200000:2.0:1787827241.504577:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000079c98697 1: 1< 0 - - > 2< 4096 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504578:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000b50b51e 00000080:00200000:2.0:1787827241.504579:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 2. 00000080:00200000:2.0:1787827241.504580:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 2 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504590:0:48388:0:(rw26.c:875:ll_write_end()) pos 8192, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504592:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000278afcbf[3 00000000f7922827 2 1 000000000104970a 0 0 2] 00000080:00200000:2.0:1787827241.504594:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c563f6e9 17ffffc0001001 3:0 ffff9a78a65210e8 2 lru 00000080:00200000:2.0:1787827241.504598:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003bbf99e6 2: 1< 0 - - > 2< 8192 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504599:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000278afcbf 00000080:00200000:2.0:1787827241.504600:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 3. 00000080:00200000:2.0:1787827241.504601:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 3 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504610:0:48388:0:(rw26.c:875:ll_write_end()) pos 12288, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504613:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005dc49113[3 00000000f7922827 2 1 000000000104970a 0 0 3] 00000080:00200000:2.0:1787827241.504615:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000786361e6 17ffffc0001001 3:0 ffff9a78a65212b8 3 lru 00000080:00200000:2.0:1787827241.504619:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c47ad184 3: 1< 0 - - > 2< 12288 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504620:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005dc49113 00000080:00200000:2.0:1787827241.504621:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 4. 00000080:00200000:2.0:1787827241.504622:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 4 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504630:0:48388:0:(rw26.c:875:ll_write_end()) pos 16384, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504632:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007951d926[3 00000000f7922827 2 1 000000000104970a 0 0 4] 00000080:00200000:2.0:1787827241.504634:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f52cc136 17ffffc0001001 3:0 ffff9a78a65211d0 4 lru 00000080:00200000:2.0:1787827241.504637:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e2f3c84a 4: 1< 0 - - > 2< 16384 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504639:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007951d926 00000080:00200000:2.0:1787827241.504640:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 5. 00000080:00200000:2.0:1787827241.504641:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 5 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504650:0:48388:0:(rw26.c:875:ll_write_end()) pos 20480, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504652:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000840773b8[3 00000000f7922827 2 1 000000000104970a 0 0 5] 00000080:00200000:2.0:1787827241.504654:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ae29b437 17ffffc0001001 3:0 ffff9a78a6521740 5 lru 00000080:00200000:2.0:1787827241.504658:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000fc356569 5: 1< 0 - - > 2< 20480 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504660:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000840773b8 00000080:00200000:2.0:1787827241.504661:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 6. 00000080:00200000:2.0:1787827241.504661:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 6 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504671:0:48388:0:(rw26.c:875:ll_write_end()) pos 24576, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504674:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000010f587f1[3 00000000f7922827 2 1 000000000104970a 0 0 6] 00000080:00200000:2.0:1787827241.504675:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000011c9ddcf 17ffffc0001001 3:0 ffff9a78a6521e80 6 lru 00000080:00200000:2.0:1787827241.504679:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006bbc73ec 6: 1< 0 - - > 2< 24576 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504681:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000010f587f1 00000080:00200000:2.0:1787827241.504681:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 7. 00000080:00200000:2.0:1787827241.504683:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 7 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504691:0:48388:0:(rw26.c:875:ll_write_end()) pos 28672, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504694:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a09da645[3 00000000f7922827 2 1 000000000104970a 0 0 7] 00000080:00200000:2.0:1787827241.504696:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000de25dff5 17ffffc0001001 3:0 ffff9a78a6521488 7 lru 00000080:00200000:2.0:1787827241.504699:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000c0225b2 7: 1< 0 - - > 2< 28672 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504700:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a09da645 00000080:00200000:2.0:1787827241.504701:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 8. 00000080:00200000:2.0:1787827241.504702:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 8 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504710:0:48388:0:(rw26.c:875:ll_write_end()) pos 32768, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504713:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005d8e6a79[3 00000000f7922827 2 1 000000000104970a 0 0 8] 00000080:00200000:2.0:1787827241.504714:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000294d12d9 17ffffc0001001 3:0 ffff9a78a6521bc8 8 lru 00000080:00200000:2.0:1787827241.504718:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000013c0d953 8: 1< 0 - - > 2< 32768 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504719:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005d8e6a79 00000080:00200000:2.0:1787827241.504720:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 9. 00000080:00200000:2.0:1787827241.504720:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 9 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504732:0:48388:0:(rw26.c:875:ll_write_end()) pos 36864, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504734:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d947bff5[3 00000000f7922827 2 1 000000000104970a 0 0 9] 00000080:00200000:2.0:1787827241.504737:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c19e27ba 17ffffc0001021 2:0 ffff9a78a6521570 9 lru 00000080:00200000:2.0:1787827241.504741:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000622bcd68 9: 1< 0 - - > 2< 36864 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504743:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d947bff5 00000080:00200000:2.0:1787827241.504744:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 10. 00000080:00200000:2.0:1787827241.504745:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 10 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504753:0:48388:0:(rw26.c:875:ll_write_end()) pos 40960, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504756:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003ee8528a[3 00000000f7922827 2 1 000000000104970a 0 0 10] 00000080:00200000:2.0:1787827241.504758:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009738b266 17ffffc0001001 3:0 ffff9a78a6521828 10 lru 00000080:00200000:2.0:1787827241.504761:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000014a7e0bc 10: 1< 0 - - > 2< 40960 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504763:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003ee8528a 00000080:00200000:2.0:1787827241.504764:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 11. 00000080:00200000:2.0:1787827241.504765:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 11 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504773:0:48388:0:(rw26.c:875:ll_write_end()) pos 45056, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504775:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d23811b4[3 00000000f7922827 2 1 000000000104970a 0 0 11] 00000080:00200000:2.0:1787827241.504777:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006e37d673 17ffffc0001001 3:0 ffff9a78a6521000 11 lru 00000080:00200000:2.0:1787827241.504780:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006ab3a8b2 11: 1< 0 - - > 2< 45056 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504781:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d23811b4 00000080:00200000:2.0:1787827241.504782:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 12. 00000080:00200000:2.0:1787827241.504783:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 12 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504791:0:48388:0:(rw26.c:875:ll_write_end()) pos 49152, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504793:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000020892711[3 00000000f7922827 2 1 000000000104970a 0 0 12] 00000080:00200000:2.0:1787827241.504795:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006702b19f 17ffffc0001001 3:0 ffff9a78a6521cb0 12 lru 00000080:00200000:2.0:1787827241.504798:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a938dcb4 12: 1< 0 - - > 2< 49152 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504799:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000020892711 00000080:00200000:2.0:1787827241.504800:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 13. 00000080:00200000:2.0:1787827241.504801:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 13 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504810:0:48388:0:(rw26.c:875:ll_write_end()) pos 53248, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504812:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000045257523[3 00000000f7922827 2 1 000000000104970a 0 0 13] 00000080:00200000:2.0:1787827241.504814:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005d3afd7f 17ffffc0001001 3:0 ffff9a78a6521ae0 13 lru 00000080:00200000:2.0:1787827241.504817:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000ded30d79 13: 1< 0 - - > 2< 53248 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504819:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000045257523 00000080:00200000:2.0:1787827241.504820:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 14. 00000080:00200000:2.0:1787827241.504821:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 14 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504849:0:48388:0:(rw26.c:875:ll_write_end()) pos 57344, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504852:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ee7ccb69[3 00000000f7922827 2 1 000000000104970a 0 0 14] 00000080:00200000:2.0:1787827241.504854:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c933dc57 17ffffc0001001 3:0 ffff9a78a6521658 14 lru 00000080:00200000:2.0:1787827241.504858:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000596d60fe 14: 1< 0 - - > 2< 57344 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504860:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ee7ccb69 00000080:00200000:2.0:1787827241.504861:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 15. 00000080:00200000:2.0:1787827241.504862:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 15 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504882:0:48388:0:(rw26.c:875:ll_write_end()) pos 61440, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504884:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000282dfa38[3 00000000f7922827 2 1 000000000104970a 0 0 15] 00000080:00200000:2.0:1787827241.504886:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000092848df 17ffffc0001001 3:0 ffff9a7886abe1d0 15 lru 00000080:00200000:2.0:1787827241.504890:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000005215de3f 15: 1< 0 - - > 2< 61440 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504892:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000282dfa38 00000080:00200000:2.0:1787827241.504893:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 16. 00000080:00200000:2.0:1787827241.504894:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 16 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504903:0:48388:0:(rw26.c:875:ll_write_end()) pos 65536, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504906:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000106d512[3 00000000f7922827 2 1 000000000104970a 0 0 16] 00000080:00200000:2.0:1787827241.504907:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000058ffc34e 17ffffc0001001 3:0 ffff9a7886abe740 16 lru 00000080:00200000:2.0:1787827241.504911:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007d966850 16: 1< 0 - - > 2< 65536 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504913:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000106d512 00000080:00200000:2.0:1787827241.504914:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 17. 00000080:00200000:2.0:1787827241.504915:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 17 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504923:0:48388:0:(rw26.c:875:ll_write_end()) pos 69632, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504925:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000068cedc89[3 00000000f7922827 2 1 000000000104970a 0 0 17] 00000080:00200000:2.0:1787827241.504927:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000867ee808 17ffffc0001001 3:0 ffff9a7886abee80 17 lru 00000080:00200000:2.0:1787827241.504931:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000362194c7 17: 1< 0 - - > 2< 69632 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504932:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000068cedc89 00000080:00200000:2.0:1787827241.504933:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 18. 00000080:00200000:2.0:1787827241.504934:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 18 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504942:0:48388:0:(rw26.c:875:ll_write_end()) pos 73728, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504945:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005cc27aef[3 00000000f7922827 2 1 000000000104970a 0 0 18] 00000080:00200000:2.0:1787827241.504947:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000003b3f482a 17ffffc0001001 3:0 ffff9a7886abe488 18 lru 00000080:00200000:2.0:1787827241.504951:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000dbe51a5f 18: 1< 0 - - > 2< 73728 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504953:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005cc27aef 00000080:00200000:2.0:1787827241.504953:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 19. 00000080:00200000:2.0:1787827241.504955:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 19 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504963:0:48388:0:(rw26.c:875:ll_write_end()) pos 77824, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504965:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000060b37dca[3 00000000f7922827 2 1 000000000104970a 0 0 19] 00000080:00200000:2.0:1787827241.504967:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000033b1decc 17ffffc0001001 3:0 ffff9a7886abebc8 19 lru 00000080:00200000:2.0:1787827241.504970:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c2f65495 19: 1< 0 - - > 2< 77824 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504972:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000060b37dca 00000080:00200000:2.0:1787827241.504972:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 20. 00000080:00200000:2.0:1787827241.504973:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 20 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.504983:0:48388:0:(rw26.c:875:ll_write_end()) pos 81920, len 4096, copied 4096 00000080:00200000:2.0:1787827241.504986:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c412005d[3 00000000f7922827 2 1 000000000104970a 0 0 20] 00000080:00200000:2.0:1787827241.504988:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f65fbb3d 17ffffc0001001 3:0 ffff9a7886abe570 20 lru 00000080:00200000:2.0:1787827241.504991:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001db12a0b 20: 1< 0 - - > 2< 81920 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.504993:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c412005d 00000080:00200000:2.0:1787827241.504994:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 21. 00000080:00200000:2.0:1787827241.504995:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 21 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505005:0:48388:0:(rw26.c:875:ll_write_end()) pos 86016, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505007:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003595a7d4[3 00000000f7922827 2 1 000000000104970a 0 0 21] 00000080:00200000:2.0:1787827241.505009:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005d855f95 17ffffc0001001 3:0 ffff9a7886abe828 21 lru 00000080:00200000:2.0:1787827241.505012:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000004c97f3fc 21: 1< 0 - - > 2< 86016 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505014:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003595a7d4 00000080:00200000:2.0:1787827241.505015:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 22. 00000080:00200000:2.0:1787827241.505016:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 22 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505024:0:48388:0:(rw26.c:875:ll_write_end()) pos 90112, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505026:0:48388:0:(rw26.c:933:ll_write_end()) page@000000008a23dca5[3 00000000f7922827 2 1 000000000104970a 0 0 22] 00000080:00200000:2.0:1787827241.505028:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c7deb9b5 17ffffc0001001 3:0 ffff9a7886abe000 22 lru 00000080:00200000:2.0:1787827241.505032:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000f76be384 22: 1< 0 - - > 2< 90112 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505033:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000008a23dca5 00000080:00200000:2.0:1787827241.505034:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 23. 00000080:00200000:2.0:1787827241.505035:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 23 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505043:0:48388:0:(rw26.c:875:ll_write_end()) pos 94208, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505046:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007920d1c3[3 00000000f7922827 2 1 000000000104970a 0 0 23] 00000080:00200000:2.0:1787827241.505048:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f87df770 17ffffc0001001 3:0 ffff9a7886abecb0 23 lru 00000080:00200000:2.0:1787827241.505051:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006ed57cd6 23: 1< 0 - - > 2< 94208 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505052:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007920d1c3 00000080:00200000:2.0:1787827241.505053:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 24. 00000080:00200000:2.0:1787827241.505054:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 24 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505065:0:48388:0:(rw26.c:875:ll_write_end()) pos 98304, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505067:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000010623789[3 00000000f7922827 2 1 000000000104970a 0 0 24] 00000080:00200000:2.0:1787827241.505069:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000aa87d424 17ffffc0001021 2:0 ffff9a7886abeae0 24 lru 00000080:00200000:2.0:1787827241.505072:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000b16649aa 24: 1< 0 - - > 2< 98304 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505074:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000010623789 00000080:00200000:2.0:1787827241.505074:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 25. 00000080:00200000:2.0:1787827241.505075:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 25 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505083:0:48388:0:(rw26.c:875:ll_write_end()) pos 102400, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505086:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000036f48520[3 00000000f7922827 2 1 000000000104970a 0 0 25] 00000080:00200000:2.0:1787827241.505087:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d3012b4c 17ffffc0001001 3:0 ffff9a7886abe658 25 lru 00000080:00200000:2.0:1787827241.505091:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006b58139b 25: 1< 0 - - > 2< 102400 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505092:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000036f48520 00000080:00200000:2.0:1787827241.505093:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 26. 00000080:00200000:2.0:1787827241.505094:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 26 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505102:0:48388:0:(rw26.c:875:ll_write_end()) pos 106496, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505104:0:48388:0:(rw26.c:933:ll_write_end()) page@000000001fe7da62[3 00000000f7922827 2 1 000000000104970a 0 0 26] 00000080:00200000:2.0:1787827241.505106:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ae7646f2 17ffffc0001001 3:0 ffff9a7886abe9f8 26 lru 00000080:00200000:2.0:1787827241.505109:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003f4689b9 26: 1< 0 - - > 2< 106496 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505110:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000001fe7da62 00000080:00200000:2.0:1787827241.505111:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 27. 00000080:00200000:2.0:1787827241.505112:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 27 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505121:0:48388:0:(rw26.c:875:ll_write_end()) pos 110592, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505123:0:48388:0:(rw26.c:933:ll_write_end()) page@000000001e49f595[3 00000000f7922827 2 1 000000000104970a 0 0 27] 00000080:00200000:2.0:1787827241.505125:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005355aac6 17ffffc0001001 3:0 ffff9a7886abe910 27 lru 00000080:00200000:2.0:1787827241.505130:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e92d63af 27: 1< 0 - - > 2< 110592 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505131:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000001e49f595 00000080:00200000:2.0:1787827241.505132:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 28. 00000080:00200000:2.0:1787827241.505133:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 28 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505142:0:48388:0:(rw26.c:875:ll_write_end()) pos 114688, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505145:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e2c5c856[3 00000000f7922827 2 1 000000000104970a 0 0 28] 00000080:00200000:2.0:1787827241.505147:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000070b43f20 17ffffc0001001 3:0 ffff9a7886abed98 28 lru 00000080:00200000:2.0:1787827241.505151:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000f6a434f5 28: 1< 0 - - > 2< 114688 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505152:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e2c5c856 00000080:00200000:2.0:1787827241.505153:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 29. 00000080:00200000:2.0:1787827241.505154:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 29 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505163:0:48388:0:(rw26.c:875:ll_write_end()) pos 118784, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505165:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d0175ae8[3 00000000f7922827 2 1 000000000104970a 0 0 29] 00000080:00200000:2.0:1787827241.505167:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b8fdc8e9 17ffffc0001001 3:0 ffff9a7886abe3a0 29 lru 00000080:00200000:2.0:1787827241.505171:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000017b7318f 29: 1< 0 - - > 2< 118784 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505172:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d0175ae8 00000080:00200000:2.0:1787827241.505173:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 30. 00000080:00200000:2.0:1787827241.505173:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 30 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505181:0:48388:0:(rw26.c:875:ll_write_end()) pos 122880, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505184:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000081b26123[3 00000000f7922827 2 1 000000000104970a 0 0 30] 00000080:00200000:2.0:1787827241.505186:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e95e1847 17ffffc0001001 3:0 ffff9a7886abe0e8 30 lru 00000080:00200000:2.0:1787827241.505189:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000fae48d05 30: 1< 0 - - > 2< 122880 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505190:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000081b26123 00000080:00200000:2.0:1787827241.505191:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 31. 00000080:00200000:2.0:1787827241.505192:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 31 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505200:0:48388:0:(rw26.c:875:ll_write_end()) pos 126976, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505202:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000860bc494[3 00000000f7922827 2 1 000000000104970a 0 0 31] 00000080:00200000:2.0:1787827241.505204:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000d9ad41b 17ffffc0001001 3:0 ffff9a7886abe2b8 31 lru 00000080:00200000:2.0:1787827241.505208:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000f7f1f09 31: 1< 0 - - > 2< 126976 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505209:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000860bc494 00000080:00200000:2.0:1787827241.505210:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 32. 00000080:00200000:2.0:1787827241.505211:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 32 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505226:0:48388:0:(rw26.c:875:ll_write_end()) pos 131072, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505229:0:48388:0:(rw26.c:933:ll_write_end()) page@000000001789bdc7[3 00000000f7922827 2 1 000000000104970a 0 0 32] 00000080:00200000:2.0:1787827241.505231:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000032b5bfb5 17ffffc0001001 3:0 ffff9a7887618ae0 32 lru 00000080:00200000:2.0:1787827241.505235:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000023b88b9f 32: 1< 0 - - > 2< 131072 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505237:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000001789bdc7 00000080:00200000:2.0:1787827241.505237:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 33. 00000080:00200000:2.0:1787827241.505238:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 33 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505247:0:48388:0:(rw26.c:875:ll_write_end()) pos 135168, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505249:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000da5decc8[3 00000000f7922827 2 1 000000000104970a 0 0 33] 00000080:00200000:2.0:1787827241.505251:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000abce0809 17ffffc0001001 3:0 ffff9a7887618658 33 lru 00000080:00200000:2.0:1787827241.505255:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000030ab1a0a 33: 1< 0 - - > 2< 135168 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505256:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000da5decc8 00000080:00200000:2.0:1787827241.505256:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 34. 00000080:00200000:2.0:1787827241.505258:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 34 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505267:0:48388:0:(rw26.c:875:ll_write_end()) pos 139264, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505270:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ca155197[3 00000000f7922827 2 1 000000000104970a 0 0 34] 00000080:00200000:2.0:1787827241.505272:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000076fa87f 17ffffc0001001 3:0 ffff9a78876189f8 34 lru 00000080:00200000:2.0:1787827241.505276:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000004b35e069 34: 1< 0 - - > 2< 139264 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505277:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ca155197 00000080:00200000:2.0:1787827241.505278:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 35. 00000080:00200000:2.0:1787827241.505279:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 35 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505287:0:48388:0:(rw26.c:875:ll_write_end()) pos 143360, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505289:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003f77a84d[3 00000000f7922827 2 1 000000000104970a 0 0 35] 00000080:00200000:2.0:1787827241.505291:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000dd4d72ec 17ffffc0001001 3:0 ffff9a7887618910 35 lru 00000080:00200000:2.0:1787827241.505294:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007ad48f21 35: 1< 0 - - > 2< 143360 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505295:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003f77a84d 00000080:00200000:2.0:1787827241.505296:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 36. 00000080:00200000:2.0:1787827241.505297:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 36 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505305:0:48388:0:(rw26.c:875:ll_write_end()) pos 147456, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505307:0:48388:0:(rw26.c:933:ll_write_end()) page@000000002cc812ee[3 00000000f7922827 2 1 000000000104970a 0 0 36] 00000080:00200000:2.0:1787827241.505310:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007124983a 17ffffc0001001 3:0 ffff9a7887618d98 36 lru 00000080:00200000:2.0:1787827241.505313:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000059034b90 36: 1< 0 - - > 2< 147456 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505315:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000002cc812ee 00000080:00200000:2.0:1787827241.505316:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 37. 00000080:00200000:2.0:1787827241.505317:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 37 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505331:0:48388:0:(rw26.c:875:ll_write_end()) pos 151552, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505336:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e9643fdb[3 00000000f7922827 2 1 000000000104970a 0 0 37] 00000080:00200000:2.0:1787827241.505340:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000537fd860 17ffffc0001001 3:0 ffff9a78876183a0 37 lru 00000080:00200000:2.0:1787827241.505345:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002f620cd4 37: 1< 0 - - > 2< 151552 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505347:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e9643fdb 00000080:00200000:2.0:1787827241.505348:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 38. 00000080:00200000:2.0:1787827241.505350:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 38 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505367:0:48388:0:(rw26.c:875:ll_write_end()) pos 155648, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505372:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000afb49a1d[3 00000000f7922827 2 1 000000000104970a 0 0 38] 00000080:00200000:2.0:1787827241.505376:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000652409c8 17ffffc0001001 3:0 ffff9a78876180e8 38 lru 00000080:00200000:2.0:1787827241.505381:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000da1f4f55 38: 1< 0 - - > 2< 155648 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505383:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000afb49a1d 00000080:00200000:2.0:1787827241.505384:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 39. 00000080:00200000:2.0:1787827241.505385:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 39 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505400:0:48388:0:(rw26.c:875:ll_write_end()) pos 159744, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505402:0:48388:0:(rw26.c:933:ll_write_end()) page@000000001d4995de[3 00000000f7922827 2 1 000000000104970a 0 0 39] 00000080:00200000:2.0:1787827241.505404:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006d3e7d0e 17ffffc0001021 2:0 ffff9a78876182b8 39 lru 00000080:00200000:2.0:1787827241.505408:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000bdf6ae14 39: 1< 0 - - > 2< 159744 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505409:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000001d4995de 00000080:00200000:2.0:1787827241.505409:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 40. 00000080:00200000:2.0:1787827241.505411:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 40 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505419:0:48388:0:(rw26.c:875:ll_write_end()) pos 163840, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505422:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000220492c8[3 00000000f7922827 2 1 000000000104970a 0 0 40] 00000080:00200000:2.0:1787827241.505423:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000eccf691f 17ffffc0001001 3:0 ffff9a78876181d0 40 lru 00000080:00200000:2.0:1787827241.505427:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003e497a97 40: 1< 0 - - > 2< 163840 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505428:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000220492c8 00000080:00200000:2.0:1787827241.505429:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 41. 00000080:00200000:2.0:1787827241.505430:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 41 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505440:0:48388:0:(rw26.c:875:ll_write_end()) pos 167936, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505443:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c7e2313f[3 00000000f7922827 2 1 000000000104970a 0 0 41] 00000080:00200000:2.0:1787827241.505445:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ec236b99 17ffffc0001001 3:0 ffff9a7887618740 41 lru 00000080:00200000:2.0:1787827241.505449:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000389073d5 41: 1< 0 - - > 2< 167936 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505451:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c7e2313f 00000080:00200000:2.0:1787827241.505453:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 42. 00000080:00200000:2.0:1787827241.505454:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 42 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505468:0:48388:0:(rw26.c:875:ll_write_end()) pos 172032, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505473:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c3f8bf74[3 00000000f7922827 2 1 000000000104970a 0 0 42] 00000080:00200000:2.0:1787827241.505476:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007c91ca89 17ffffc0001001 3:0 ffff9a7887618e80 42 lru 00000080:00200000:2.0:1787827241.505483:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000deb092f5 42: 1< 0 - - > 2< 172032 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505485:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c3f8bf74 00000080:00200000:2.0:1787827241.505486:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 43. 00000080:00200000:2.0:1787827241.505488:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 43 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505514:0:48388:0:(rw26.c:875:ll_write_end()) pos 176128, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505518:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007a496cb2[3 00000000f7922827 2 1 000000000104970a 0 0 43] 00000080:00200000:2.0:1787827241.505520:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e732b9ad 17ffffc0001001 3:0 ffff9a7887618488 43 lru 00000080:00200000:2.0:1787827241.505524:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000048ad2a8f 43: 1< 0 - - > 2< 176128 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505525:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007a496cb2 00000080:00200000:2.0:1787827241.505526:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 44. 00000080:00200000:2.0:1787827241.505528:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 44 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505538:0:48388:0:(rw26.c:875:ll_write_end()) pos 180224, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505541:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000b333ebe1[3 00000000f7922827 2 1 000000000104970a 0 0 44] 00000080:00200000:2.0:1787827241.505543:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000fcdb09c5 17ffffc0001001 3:0 ffff9a7887618bc8 44 lru 00000080:00200000:2.0:1787827241.505547:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000cf753ce 44: 1< 0 - - > 2< 180224 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505549:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000b333ebe1 00000080:00200000:2.0:1787827241.505549:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 45. 00000080:00200000:2.0:1787827241.505550:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 45 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505558:0:48388:0:(rw26.c:875:ll_write_end()) pos 184320, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505561:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000014bff268[3 00000000f7922827 2 1 000000000104970a 0 0 45] 00000080:00200000:2.0:1787827241.505563:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f703174f 17ffffc0001001 3:0 ffff9a7887618570 45 lru 00000080:00200000:2.0:1787827241.505567:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000590423a4 45: 1< 0 - - > 2< 184320 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505569:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000014bff268 00000080:00200000:2.0:1787827241.505570:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 46. 00000080:00200000:2.0:1787827241.505571:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 46 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505579:0:48388:0:(rw26.c:875:ll_write_end()) pos 188416, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505581:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000785c407f[3 00000000f7922827 2 1 000000000104970a 0 0 46] 00000080:00200000:2.0:1787827241.505583:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000041efdd8 17ffffc0001001 3:0 ffff9a7887618828 46 lru 00000080:00200000:2.0:1787827241.505586:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000014338ac9 46: 1< 0 - - > 2< 188416 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505587:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000785c407f 00000080:00200000:2.0:1787827241.505588:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 47. 00000080:00200000:2.0:1787827241.505589:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 47 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505602:0:48388:0:(rw26.c:875:ll_write_end()) pos 192512, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505606:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000016cc50cf[3 00000000f7922827 2 1 000000000104970a 0 0 47] 00000080:00200000:2.0:1787827241.505609:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000bf74fe96 17ffffc0001001 3:0 ffff9a7887618000 47 lru 00000080:00200000:2.0:1787827241.505614:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000492b7cfc 47: 1< 0 - - > 2< 192512 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505616:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000016cc50cf 00000080:00200000:2.0:1787827241.505617:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 48. 00000080:00200000:2.0:1787827241.505619:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 48 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505636:0:48388:0:(rw26.c:875:ll_write_end()) pos 196608, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505641:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003b82c4dc[3 00000000f7922827 2 1 000000000104970a 0 0 48] 00000080:00200000:2.0:1787827241.505644:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000003458ac71 17ffffc0001001 3:0 ffff9a7887618cb0 48 lru 00000080:00200000:2.0:1787827241.505650:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003d1201a0 48: 1< 0 - - > 2< 196608 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505652:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003b82c4dc 00000080:00200000:2.0:1787827241.505654:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 49. 00000080:00200000:2.0:1787827241.505656:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 49 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505678:0:48388:0:(rw26.c:875:ll_write_end()) pos 200704, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505683:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000043e60004[3 00000000f7922827 2 1 000000000104970a 0 0 49] 00000080:00200000:2.0:1787827241.505687:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000dce79c5f 17ffffc0001001 3:0 ffff9a78909b7910 49 lru 00000080:00200000:2.0:1787827241.505693:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000ab2d3eb4 49: 1< 0 - - > 2< 200704 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505696:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000043e60004 00000080:00200000:2.0:1787827241.505698:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 50. 00000080:00200000:2.0:1787827241.505700:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 50 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505717:0:48388:0:(rw26.c:875:ll_write_end()) pos 204800, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505722:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c9a2bfb4[3 00000000f7922827 2 1 000000000104970a 0 0 50] 00000080:00200000:2.0:1787827241.505726:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e605a4e8 17ffffc0001001 3:0 ffff9a78909b7d98 50 lru 00000080:00200000:2.0:1787827241.505732:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000cb80e845 50: 1< 0 - - > 2< 204800 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505734:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c9a2bfb4 00000080:00200000:2.0:1787827241.505736:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 51. 00000080:00200000:2.0:1787827241.505738:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 51 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505754:0:48388:0:(rw26.c:875:ll_write_end()) pos 208896, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505760:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000391671d6[3 00000000f7922827 2 1 000000000104970a 0 0 51] 00000080:00200000:2.0:1787827241.505764:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000004780cf4a 17ffffc0001001 3:0 ffff9a78909b73a0 51 lru 00000080:00200000:2.0:1787827241.505771:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000fcc56c43 51: 1< 0 - - > 2< 208896 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505774:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000391671d6 00000080:00200000:2.0:1787827241.505776:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 52. 00000080:00200000:2.0:1787827241.505778:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 52 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505792:0:48388:0:(rw26.c:875:ll_write_end()) pos 212992, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505797:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a14f3377[3 00000000f7922827 2 1 000000000104970a 0 0 52] 00000080:00200000:2.0:1787827241.505801:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000533da593 17ffffc0001001 3:0 ffff9a78909b70e8 52 lru 00000080:00200000:2.0:1787827241.505807:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000091799b9f 52: 1< 0 - - > 2< 212992 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505810:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a14f3377 00000080:00200000:2.0:1787827241.505811:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 53. 00000080:00200000:2.0:1787827241.505813:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 53 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505827:0:48388:0:(rw26.c:875:ll_write_end()) pos 217088, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505857:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000001bc6095[3 00000000f7922827 2 1 000000000104970a 0 0 53] 00000080:00200000:2.0:1787827241.505860:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000003033dacb 17ffffc0001001 3:0 ffff9a78909b72b8 53 lru 00000080:00200000:2.0:1787827241.505867:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c72b1f5c 53: 1< 0 - - > 2< 217088 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505869:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000001bc6095 00000080:00200000:2.0:1787827241.505871:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 54. 00000080:00200000:2.0:1787827241.505873:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 54 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505895:0:48388:0:(rw26.c:875:ll_write_end()) pos 221184, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505900:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000081d51b04[3 00000000f7922827 2 1 000000000104970a 0 0 54] 00000080:00200000:2.0:1787827241.505904:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000359eefeb 17ffffc0001021 2:0 ffff9a78909b71d0 54 lru 00000080:00200000:2.0:1787827241.505911:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009bc1dd7a 54: 1< 0 - - > 2< 221184 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505913:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000081d51b04 00000080:00200000:2.0:1787827241.505915:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 55. 00000080:00200000:2.0:1787827241.505917:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 55 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505936:0:48388:0:(rw26.c:875:ll_write_end()) pos 225280, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505940:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000092d2cd2a[3 00000000f7922827 2 1 000000000104970a 0 0 55] 00000080:00200000:2.0:1787827241.505944:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006046cc04 17ffffc0001001 3:0 ffff9a78909b7740 55 lru 00000080:00200000:2.0:1787827241.505950:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000690bff02 55: 1< 0 - - > 2< 225280 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505953:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000092d2cd2a 00000080:00200000:2.0:1787827241.505955:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 56. 00000080:00200000:2.0:1787827241.505956:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 56 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505968:0:48388:0:(rw26.c:875:ll_write_end()) pos 229376, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505971:0:48388:0:(rw26.c:933:ll_write_end()) page@000000002d6bf909[3 00000000f7922827 2 1 000000000104970a 0 0 56] 00000080:00200000:2.0:1787827241.505973:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000208d7f01 17ffffc0001001 3:0 ffff9a78909b7e80 56 lru 00000080:00200000:2.0:1787827241.505977:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000065b1848f 56: 1< 0 - - > 2< 229376 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505979:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000002d6bf909 00000080:00200000:2.0:1787827241.505979:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 57. 00000080:00200000:2.0:1787827241.505981:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 57 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.505989:0:48388:0:(rw26.c:875:ll_write_end()) pos 233472, len 4096, copied 4096 00000080:00200000:2.0:1787827241.505992:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000cdaa7756[3 00000000f7922827 2 1 000000000104970a 0 0 57] 00000080:00200000:2.0:1787827241.505993:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e3c426ab 17ffffc0001001 3:0 ffff9a78909b7488 57 lru 00000080:00200000:2.0:1787827241.505997:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001a2e4a59 57: 1< 0 - - > 2< 233472 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.505999:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000cdaa7756 00000080:00200000:2.0:1787827241.506000:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 58. 00000080:00200000:2.0:1787827241.506001:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 58 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506009:0:48388:0:(rw26.c:875:ll_write_end()) pos 237568, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506012:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000249fc171[3 00000000f7922827 2 1 000000000104970a 0 0 58] 00000080:00200000:2.0:1787827241.506013:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000890cbdfb 17ffffc0001001 3:0 ffff9a78909b7bc8 58 lru 00000080:00200000:2.0:1787827241.506017:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000084f245d4 58: 1< 0 - - > 2< 237568 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506018:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000249fc171 00000080:00200000:2.0:1787827241.506019:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 59. 00000080:00200000:2.0:1787827241.506020:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 59 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506029:0:48388:0:(rw26.c:875:ll_write_end()) pos 241664, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506031:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000033b969be[3 00000000f7922827 2 1 000000000104970a 0 0 59] 00000080:00200000:2.0:1787827241.506033:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000980aacdf 17ffffc0001001 3:0 ffff9a78909b7570 59 lru 00000080:00200000:2.0:1787827241.506037:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d18c1842 59: 1< 0 - - > 2< 241664 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506038:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000033b969be 00000080:00200000:2.0:1787827241.506039:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 60. 00000080:00200000:2.0:1787827241.506041:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 60 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506049:0:48388:0:(rw26.c:875:ll_write_end()) pos 245760, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506051:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000cb1e0cb2[3 00000000f7922827 2 1 000000000104970a 0 0 60] 00000080:00200000:2.0:1787827241.506053:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009b57e638 17ffffc0001001 3:0 ffff9a78909b7828 60 lru 00000080:00200000:2.0:1787827241.506056:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000392dcf13 60: 1< 0 - - > 2< 245760 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506057:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000cb1e0cb2 00000080:00200000:2.0:1787827241.506058:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 61. 00000080:00200000:2.0:1787827241.506059:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 61 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506067:0:48388:0:(rw26.c:875:ll_write_end()) pos 249856, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506069:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000066f771a2[3 00000000f7922827 2 1 000000000104970a 0 0 61] 00000080:00200000:2.0:1787827241.506071:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005f0ddbc5 17ffffc0001001 3:0 ffff9a78909b7000 61 lru 00000080:00200000:2.0:1787827241.506074:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d4402307 61: 1< 0 - - > 2< 249856 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506076:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000066f771a2 00000080:00200000:2.0:1787827241.506076:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 62. 00000080:00200000:2.0:1787827241.506077:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 62 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506086:0:48388:0:(rw26.c:875:ll_write_end()) pos 253952, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506089:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e658958f[3 00000000f7922827 2 1 000000000104970a 0 0 62] 00000080:00200000:2.0:1787827241.506090:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000030b91f1e 17ffffc0001001 3:0 ffff9a78909b7cb0 62 lru 00000080:00200000:2.0:1787827241.506094:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000090f8188b 62: 1< 0 - - > 2< 253952 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506096:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e658958f 00000080:00200000:2.0:1787827241.506097:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 63. 00000080:00200000:2.0:1787827241.506098:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 63 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506106:0:48388:0:(rw26.c:875:ll_write_end()) pos 258048, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506108:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000061f95366[3 00000000f7922827 2 1 000000000104970a 0 0 63] 00000080:00200000:2.0:1787827241.506111:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000081edf7a 17ffffc0001001 3:0 ffff9a78909b7ae0 63 lru 00000080:00200000:2.0:1787827241.506114:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000082ccb1cb 63: 1< 0 - - > 2< 258048 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506116:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000061f95366 00000080:00200000:2.0:1787827241.506117:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 64. 00000080:00200000:2.0:1787827241.506118:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 64 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506130:0:48388:0:(rw26.c:875:ll_write_end()) pos 262144, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506132:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005283c6a4[3 00000000f7922827 2 1 000000000104970a 0 0 64] 00000080:00200000:2.0:1787827241.506134:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000a2f1b762 17ffffc0001001 3:0 ffff9a78909b7658 64 lru 00000080:00200000:2.0:1787827241.506138:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000603d2ec1 64: 1< 0 - - > 2< 262144 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506139:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005283c6a4 00000080:00200000:2.0:1787827241.506140:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 65. 00000080:00200000:2.0:1787827241.506141:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 65 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506150:0:48388:0:(rw26.c:875:ll_write_end()) pos 266240, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506152:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000925d8af9[3 00000000f7922827 2 1 000000000104970a 0 0 65] 00000080:00200000:2.0:1787827241.506154:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000004ad70f01 17ffffc0001001 3:0 ffff9a78909b79f8 65 lru 00000080:00200000:2.0:1787827241.506157:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002e4c04ea 65: 1< 0 - - > 2< 266240 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506159:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000925d8af9 00000080:00200000:2.0:1787827241.506160:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 66. 00000080:00200000:2.0:1787827241.506161:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 66 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506176:0:48388:0:(rw26.c:875:ll_write_end()) pos 270336, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506179:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d71e2c77[3 00000000f7922827 2 1 000000000104970a 0 0 66] 00000080:00200000:2.0:1787827241.506181:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ec724101 17ffffc0001001 3:0 ffff9a7887a84000 66 lru 00000080:00200000:2.0:1787827241.506184:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000054d00dc2 66: 1< 0 - - > 2< 270336 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506186:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d71e2c77 00000080:00200000:2.0:1787827241.506187:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 67. 00000080:00200000:2.0:1787827241.506188:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 67 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506196:0:48388:0:(rw26.c:875:ll_write_end()) pos 274432, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506198:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ce80a565[3 00000000f7922827 2 1 000000000104970a 0 0 67] 00000080:00200000:2.0:1787827241.506200:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000372802f 17ffffc0001001 3:0 ffff9a7887a84cb0 67 lru 00000080:00200000:2.0:1787827241.506203:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000004110da39 67: 1< 0 - - > 2< 274432 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506205:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ce80a565 00000080:00200000:2.0:1787827241.506206:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 68. 00000080:00200000:2.0:1787827241.506207:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 68 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506216:0:48388:0:(rw26.c:875:ll_write_end()) pos 278528, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506218:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000bdecc312[3 00000000f7922827 2 1 000000000104970a 0 0 68] 00000080:00200000:2.0:1787827241.506220:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000080e06cb2 17ffffc0001001 3:0 ffff9a7887a84ae0 68 lru 00000080:00200000:2.0:1787827241.506223:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000012e40545 68: 1< 0 - - > 2< 278528 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506224:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000bdecc312 00000080:00200000:2.0:1787827241.506225:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 69. 00000080:00200000:2.0:1787827241.506227:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 69 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506238:0:48388:0:(rw26.c:875:ll_write_end()) pos 282624, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506241:0:48388:0:(rw26.c:933:ll_write_end()) page@000000004584d3a6[3 00000000f7922827 2 1 000000000104970a 0 0 69] 00000080:00200000:2.0:1787827241.506243:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000003ba82f08 17ffffc0001021 2:0 ffff9a7887a84658 69 lru 00000080:00200000:2.0:1787827241.506247:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000853a4aba 69: 1< 0 - - > 2< 282624 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506249:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000004584d3a6 00000080:00200000:2.0:1787827241.506250:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 70. 00000080:00200000:2.0:1787827241.506251:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 70 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506259:0:48388:0:(rw26.c:875:ll_write_end()) pos 286720, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506261:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000cec338f2[3 00000000f7922827 2 1 000000000104970a 0 0 70] 00000080:00200000:2.0:1787827241.506263:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002f5908bb 17ffffc0001001 3:0 ffff9a7887a849f8 70 lru 00000080:00200000:2.0:1787827241.506267:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e9981a4d 70: 1< 0 - - > 2< 286720 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506268:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000cec338f2 00000080:00200000:2.0:1787827241.506269:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 71. 00000080:00200000:2.0:1787827241.506270:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 71 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506278:0:48388:0:(rw26.c:875:ll_write_end()) pos 290816, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506281:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005f54a90f[3 00000000f7922827 2 1 000000000104970a 0 0 71] 00000080:00200000:2.0:1787827241.506283:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000fad42cbc 17ffffc0001001 3:0 ffff9a7887a84910 71 lru 00000080:00200000:2.0:1787827241.506286:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000468af345 71: 1< 0 - - > 2< 290816 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506287:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005f54a90f 00000080:00200000:2.0:1787827241.506288:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 72. 00000080:00200000:2.0:1787827241.506289:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 72 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506297:0:48388:0:(rw26.c:875:ll_write_end()) pos 294912, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506299:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000698d1cba[3 00000000f7922827 2 1 000000000104970a 0 0 72] 00000080:00200000:2.0:1787827241.506302:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000021aa711c 17ffffc0001001 3:0 ffff9a7887a84d98 72 lru 00000080:00200000:2.0:1787827241.506305:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000013889b45 72: 1< 0 - - > 2< 294912 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506306:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000698d1cba 00000080:00200000:2.0:1787827241.506307:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 73. 00000080:00200000:2.0:1787827241.506308:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 73 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506317:0:48388:0:(rw26.c:875:ll_write_end()) pos 299008, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506321:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000087d8e83a[3 00000000f7922827 2 1 000000000104970a 0 0 73] 00000080:00200000:2.0:1787827241.506324:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c1cb1fd9 17ffffc0001001 3:0 ffff9a7887a843a0 73 lru 00000080:00200000:2.0:1787827241.506330:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000635dd7d9 73: 1< 0 - - > 2< 299008 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506332:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000087d8e83a 00000080:00200000:2.0:1787827241.506333:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 74. 00000080:00200000:2.0:1787827241.506335:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 74 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506351:0:48388:0:(rw26.c:875:ll_write_end()) pos 303104, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506355:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000061b7cade[3 00000000f7922827 2 1 000000000104970a 0 0 74] 00000080:00200000:2.0:1787827241.506358:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000539ec95c 17ffffc0001001 3:0 ffff9a7887a840e8 74 lru 00000080:00200000:2.0:1787827241.506365:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000059d9bce 74: 1< 0 - - > 2< 303104 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506367:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000061b7cade 00000080:00200000:2.0:1787827241.506368:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 75. 00000080:00200000:2.0:1787827241.506371:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 75 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506384:0:48388:0:(rw26.c:875:ll_write_end()) pos 307200, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506387:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ff035a3b[3 00000000f7922827 2 1 000000000104970a 0 0 75] 00000080:00200000:2.0:1787827241.506389:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002bffccbf 17ffffc0001001 3:0 ffff9a7887a842b8 75 lru 00000080:00200000:2.0:1787827241.506393:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006f732fcb 75: 1< 0 - - > 2< 307200 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506395:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ff035a3b 00000080:00200000:2.0:1787827241.506396:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 76. 00000080:00200000:2.0:1787827241.506397:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 76 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506407:0:48388:0:(rw26.c:875:ll_write_end()) pos 311296, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506409:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000bbe0ac13[3 00000000f7922827 2 1 000000000104970a 0 0 76] 00000080:00200000:2.0:1787827241.506411:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006e774beb 17ffffc0001001 3:0 ffff9a7887a841d0 76 lru 00000080:00200000:2.0:1787827241.506415:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000a62cf45 76: 1< 0 - - > 2< 311296 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506417:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000bbe0ac13 00000080:00200000:2.0:1787827241.506418:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 77. 00000080:00200000:2.0:1787827241.506419:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 77 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506428:0:48388:0:(rw26.c:875:ll_write_end()) pos 315392, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506431:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006d21ba2b[3 00000000f7922827 2 1 000000000104970a 0 0 77] 00000080:00200000:2.0:1787827241.506432:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000001ffbd49b 17ffffc0001001 3:0 ffff9a7887a84740 77 lru 00000080:00200000:2.0:1787827241.506436:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009cf52b64 77: 1< 0 - - > 2< 315392 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506437:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006d21ba2b 00000080:00200000:2.0:1787827241.506438:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 78. 00000080:00200000:2.0:1787827241.506439:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 78 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506447:0:48388:0:(rw26.c:875:ll_write_end()) pos 319488, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506450:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000030e8e946[3 00000000f7922827 2 1 000000000104970a 0 0 78] 00000080:00200000:2.0:1787827241.506451:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000008f346a61 17ffffc0001001 3:0 ffff9a7887a84e80 78 lru 00000080:00200000:2.0:1787827241.506455:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000014318a6c 78: 1< 0 - - > 2< 319488 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506456:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000030e8e946 00000080:00200000:2.0:1787827241.506457:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 79. 00000080:00200000:2.0:1787827241.506458:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 79 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506465:0:48388:0:(rw26.c:875:ll_write_end()) pos 323584, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506468:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000960df6eb[3 00000000f7922827 2 1 000000000104970a 0 0 79] 00000080:00200000:2.0:1787827241.506470:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b019c7a1 17ffffc0001001 3:0 ffff9a7887a84488 79 lru 00000080:00200000:2.0:1787827241.506473:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007b2d70be 79: 1< 0 - - > 2< 323584 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506474:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000960df6eb 00000080:00200000:2.0:1787827241.506475:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 80. 00000080:00200000:2.0:1787827241.506476:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 80 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506484:0:48388:0:(rw26.c:875:ll_write_end()) pos 327680, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506487:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000094cf60e4[3 00000000f7922827 2 1 000000000104970a 0 0 80] 00000080:00200000:2.0:1787827241.506488:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e27136a2 17ffffc0001001 3:0 ffff9a7887a84bc8 80 lru 00000080:00200000:2.0:1787827241.506492:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000b2188d74 80: 1< 0 - - > 2< 327680 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506493:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000094cf60e4 00000080:00200000:2.0:1787827241.506493:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 81. 00000080:00200000:2.0:1787827241.506495:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 81 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506517:0:48388:0:(rw26.c:875:ll_write_end()) pos 331776, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506521:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a68857ef[3 00000000f7922827 2 1 000000000104970a 0 0 81] 00000080:00200000:2.0:1787827241.506523:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000019513cbe 17ffffc0001001 3:0 ffff9a7887a84570 81 lru 00000080:00200000:2.0:1787827241.506527:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007c115ca3 81: 1< 0 - - > 2< 331776 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506529:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a68857ef 00000080:00200000:2.0:1787827241.506530:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 82. 00000080:00200000:2.0:1787827241.506531:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 82 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506543:0:48388:0:(rw26.c:875:ll_write_end()) pos 335872, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506545:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000066245527[3 00000000f7922827 2 1 000000000104970a 0 0 82] 00000080:00200000:2.0:1787827241.506547:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000bacadf36 17ffffc0001001 3:0 ffff9a7887a84828 82 lru 00000080:00200000:2.0:1787827241.506551:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000a9b9834 82: 1< 0 - - > 2< 335872 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506553:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000066245527 00000080:00200000:2.0:1787827241.506554:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 83. 00000080:00200000:2.0:1787827241.506555:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 83 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506573:0:48388:0:(rw26.c:875:ll_write_end()) pos 339968, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506575:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ef202131[3 00000000f7922827 2 1 000000000104970a 0 0 83] 00000080:00200000:2.0:1787827241.506577:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e445e642 17ffffc0001001 3:0 ffff9a789978cd98 83 lru 00000080:00200000:2.0:1787827241.506580:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000064700017 83: 1< 0 - - > 2< 339968 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506582:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ef202131 00000080:00200000:2.0:1787827241.506583:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 84. 00000080:00200000:2.0:1787827241.506584:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 84 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506595:0:48388:0:(rw26.c:875:ll_write_end()) pos 344064, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506598:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007875f355[3 00000000f7922827 2 1 000000000104970a 0 0 84] 00000080:00200000:2.0:1787827241.506600:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000008d590839 17ffffc0001021 2:0 ffff9a789978c3a0 84 lru 00000080:00200000:2.0:1787827241.506604:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002a9fe66a 84: 1< 0 - - > 2< 344064 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506606:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007875f355 00000080:00200000:2.0:1787827241.506606:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 85. 00000080:00200000:2.0:1787827241.506607:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 85 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506616:0:48388:0:(rw26.c:875:ll_write_end()) pos 348160, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506618:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000fb59a90f[3 00000000f7922827 2 1 000000000104970a 0 0 85] 00000080:00200000:2.0:1787827241.506620:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000a1d089c7 17ffffc0001001 3:0 ffff9a789978c0e8 85 lru 00000080:00200000:2.0:1787827241.506624:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009e24796a 85: 1< 0 - - > 2< 348160 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506625:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000fb59a90f 00000080:00200000:2.0:1787827241.506626:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 86. 00000080:00200000:2.0:1787827241.506628:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 86 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506636:0:48388:0:(rw26.c:875:ll_write_end()) pos 352256, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506639:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005a3e5718[3 00000000f7922827 2 1 000000000104970a 0 0 86] 00000080:00200000:2.0:1787827241.506641:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009001b501 17ffffc0001001 3:0 ffff9a789978c2b8 86 lru 00000080:00200000:2.0:1787827241.506644:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009a43bc85 86: 1< 0 - - > 2< 352256 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506645:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005a3e5718 00000080:00200000:2.0:1787827241.506646:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 87. 00000080:00200000:2.0:1787827241.506647:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 87 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506655:0:48388:0:(rw26.c:875:ll_write_end()) pos 356352, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506657:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c268dde5[3 00000000f7922827 2 1 000000000104970a 0 0 87] 00000080:00200000:2.0:1787827241.506659:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c2bf8bf6 17ffffc0001001 3:0 ffff9a789978c1d0 87 lru 00000080:00200000:2.0:1787827241.506662:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000dadc92a1 87: 1< 0 - - > 2< 356352 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506663:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c268dde5 00000080:00200000:2.0:1787827241.506664:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 88. 00000080:00200000:2.0:1787827241.506665:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 88 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506673:0:48388:0:(rw26.c:875:ll_write_end()) pos 360448, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506676:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006fe32adf[3 00000000f7922827 2 1 000000000104970a 0 0 88] 00000080:00200000:2.0:1787827241.506678:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005ba8ca4c 17ffffc0001001 3:0 ffff9a789978c740 88 lru 00000080:00200000:2.0:1787827241.506681:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000f7930a6 88: 1< 0 - - > 2< 360448 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506682:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006fe32adf 00000080:00200000:2.0:1787827241.506683:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 89. 00000080:00200000:2.0:1787827241.506684:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 89 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506692:0:48388:0:(rw26.c:875:ll_write_end()) pos 364544, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506694:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007c58c274[3 00000000f7922827 2 1 000000000104970a 0 0 89] 00000080:00200000:2.0:1787827241.506696:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000041dde10 17ffffc0001001 3:0 ffff9a789978ce80 89 lru 00000080:00200000:2.0:1787827241.506699:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000df23f2f4 89: 1< 0 - - > 2< 364544 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506700:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007c58c274 00000080:00200000:2.0:1787827241.506701:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 90. 00000080:00200000:2.0:1787827241.506702:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 90 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506712:0:48388:0:(rw26.c:875:ll_write_end()) pos 368640, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506714:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000025c8624e[3 00000000f7922827 2 1 000000000104970a 0 0 90] 00000080:00200000:2.0:1787827241.506716:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ac9deb80 17ffffc0001001 3:0 ffff9a789978c488 90 lru 00000080:00200000:2.0:1787827241.506721:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007c3e1a8c 90: 1< 0 - - > 2< 368640 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506724:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000025c8624e 00000080:00200000:2.0:1787827241.506725:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 91. 00000080:00200000:2.0:1787827241.506727:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 91 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506742:0:48388:0:(rw26.c:875:ll_write_end()) pos 372736, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506747:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000a7dea8e[3 00000000f7922827 2 1 000000000104970a 0 0 91] 00000080:00200000:2.0:1787827241.506750:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e955920c 17ffffc0001001 3:0 ffff9a789978cbc8 91 lru 00000080:00200000:2.0:1787827241.506756:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000090a0f8c5 91: 1< 0 - - > 2< 372736 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506758:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000a7dea8e 00000080:00200000:2.0:1787827241.506760:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 92. 00000080:00200000:2.0:1787827241.506762:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 92 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506780:0:48388:0:(rw26.c:875:ll_write_end()) pos 376832, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506785:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d305c91e[3 00000000f7922827 2 1 000000000104970a 0 0 92] 00000080:00200000:2.0:1787827241.506789:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000042ef516b 17ffffc0001001 3:0 ffff9a789978c570 92 lru 00000080:00200000:2.0:1787827241.506794:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000024c962ed 92: 1< 0 - - > 2< 376832 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506796:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d305c91e 00000080:00200000:2.0:1787827241.506797:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 93. 00000080:00200000:2.0:1787827241.506799:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 93 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506811:0:48388:0:(rw26.c:875:ll_write_end()) pos 380928, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506814:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005817d38d[3 00000000f7922827 2 1 000000000104970a 0 0 93] 00000080:00200000:2.0:1787827241.506816:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000de58d718 17ffffc0001001 3:0 ffff9a789978c828 93 lru 00000080:00200000:2.0:1787827241.506820:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003770dc0d 93: 1< 0 - - > 2< 380928 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506821:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005817d38d 00000080:00200000:2.0:1787827241.506822:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 94. 00000080:00200000:2.0:1787827241.506823:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 94 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506850:0:48388:0:(rw26.c:875:ll_write_end()) pos 385024, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506856:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006c785c3c[3 00000000f7922827 2 1 000000000104970a 0 0 94] 00000080:00200000:2.0:1787827241.506859:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000001b772f8a 17ffffc0001001 3:0 ffff9a789978c000 94 lru 00000080:00200000:2.0:1787827241.506866:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000ef817577 94: 1< 0 - - > 2< 385024 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506869:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006c785c3c 00000080:00200000:2.0:1787827241.506872:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 95. 00000080:00200000:2.0:1787827241.506875:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 95 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506893:0:48388:0:(rw26.c:875:ll_write_end()) pos 389120, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506899:0:48388:0:(rw26.c:933:ll_write_end()) page@000000008478862c[3 00000000f7922827 2 1 000000000104970a 0 0 95] 00000080:00200000:2.0:1787827241.506902:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000081a919f8 17ffffc0001001 3:0 ffff9a789978ccb0 95 lru 00000080:00200000:2.0:1787827241.506908:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000005c0934c3 95: 1< 0 - - > 2< 389120 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506910:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000008478862c 00000080:00200000:2.0:1787827241.506911:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 96. 00000080:00200000:2.0:1787827241.506914:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 96 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506929:0:48388:0:(rw26.c:875:ll_write_end()) pos 393216, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506934:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000087fc3fe6[3 00000000f7922827 2 1 000000000104970a 0 0 96] 00000080:00200000:2.0:1787827241.506938:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000889ba366 17ffffc0001001 3:0 ffff9a789978cae0 96 lru 00000080:00200000:2.0:1787827241.506943:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000f7ca22af 96: 1< 0 - - > 2< 393216 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506946:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000087fc3fe6 00000080:00200000:2.0:1787827241.506947:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 97. 00000080:00200000:2.0:1787827241.506949:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 97 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506966:0:48388:0:(rw26.c:875:ll_write_end()) pos 397312, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506971:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000013ed3015[3 00000000f7922827 2 1 000000000104970a 0 0 97] 00000080:00200000:2.0:1787827241.506974:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e04ca142 17ffffc0001001 3:0 ffff9a789978c658 97 lru 00000080:00200000:2.0:1787827241.506978:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c78a13c9 97: 1< 0 - - > 2< 397312 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.506980:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000013ed3015 00000080:00200000:2.0:1787827241.506981:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 98. 00000080:00200000:2.0:1787827241.506982:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 98 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.506993:0:48388:0:(rw26.c:875:ll_write_end()) pos 401408, len 4096, copied 4096 00000080:00200000:2.0:1787827241.506995:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c4f8dbb7[3 00000000f7922827 2 1 000000000104970a 0 0 98] 00000080:00200000:2.0:1787827241.506997:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000016537420 17ffffc0001001 3:0 ffff9a789978c9f8 98 lru 00000080:00200000:2.0:1787827241.507001:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001d881c78 98: 1< 0 - - > 2< 401408 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507003:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c4f8dbb7 00000080:00200000:2.0:1787827241.507003:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 99. 00000080:00200000:2.0:1787827241.507004:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 99 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507015:0:48388:0:(rw26.c:875:ll_write_end()) pos 405504, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507018:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000058cd1d13[3 00000000f7922827 2 1 000000000104970a 0 0 99] 00000080:00200000:2.0:1787827241.507020:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000003bae332a 17ffffc0001021 2:0 ffff9a789978c910 99 lru 00000080:00200000:2.0:1787827241.507024:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000cdef9814 99: 1< 0 - - > 2< 405504 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507026:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000058cd1d13 00000080:00200000:2.0:1787827241.507027:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 100. 00000080:00200000:2.0:1787827241.507028:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 100 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507043:0:48388:0:(rw26.c:875:ll_write_end()) pos 409600, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507046:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000082cc6c72[3 00000000f7922827 2 1 000000000104970a 0 0 100] 00000080:00200000:2.0:1787827241.507048:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000768c5fdb 17ffffc0001001 3:0 ffff9a78835e49f8 100 lru 00000080:00200000:2.0:1787827241.507051:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000ef13f162 100: 1< 0 - - > 2< 409600 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507053:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000082cc6c72 00000080:00200000:2.0:1787827241.507054:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 101. 00000080:00200000:2.0:1787827241.507055:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 101 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507064:0:48388:0:(rw26.c:875:ll_write_end()) pos 413696, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507067:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000f05ec2b[3 00000000f7922827 2 1 000000000104970a 0 0 101] 00000080:00200000:2.0:1787827241.507069:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f2c2d75d 17ffffc0001001 3:0 ffff9a78835e4910 101 lru 00000080:00200000:2.0:1787827241.507073:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000005b6db48e 101: 1< 0 - - > 2< 413696 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507074:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000f05ec2b 00000080:00200000:2.0:1787827241.507075:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 102. 00000080:00200000:2.0:1787827241.507076:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 102 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507084:0:48388:0:(rw26.c:875:ll_write_end()) pos 417792, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507086:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007df06542[3 00000000f7922827 2 1 000000000104970a 0 0 102] 00000080:00200000:2.0:1787827241.507088:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000295e730d 17ffffc0001001 3:0 ffff9a78835e4d98 102 lru 00000080:00200000:2.0:1787827241.507092:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000188bd632 102: 1< 0 - - > 2< 417792 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507093:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007df06542 00000080:00200000:2.0:1787827241.507094:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 103. 00000080:00200000:2.0:1787827241.507095:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 103 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507103:0:48388:0:(rw26.c:875:ll_write_end()) pos 421888, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507106:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005655ff97[3 00000000f7922827 2 1 000000000104970a 0 0 103] 00000080:00200000:2.0:1787827241.507108:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000873d7375 17ffffc0001001 3:0 ffff9a78835e43a0 103 lru 00000080:00200000:2.0:1787827241.507111:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a68eecae 103: 1< 0 - - > 2< 421888 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507114:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005655ff97 00000080:00200000:2.0:1787827241.507114:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 104. 00000080:00200000:2.0:1787827241.507115:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 104 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507125:0:48388:0:(rw26.c:875:ll_write_end()) pos 425984, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507128:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a608fae6[3 00000000f7922827 2 1 000000000104970a 0 0 104] 00000080:00200000:2.0:1787827241.507129:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000060293b6d 17ffffc0001001 3:0 ffff9a78835e40e8 104 lru 00000080:00200000:2.0:1787827241.507133:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000ace27a8 104: 1< 0 - - > 2< 425984 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507134:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a608fae6 00000080:00200000:2.0:1787827241.507134:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 105. 00000080:00200000:2.0:1787827241.507135:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 105 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507144:0:48388:0:(rw26.c:875:ll_write_end()) pos 430080, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507146:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006510a877[3 00000000f7922827 2 1 000000000104970a 0 0 105] 00000080:00200000:2.0:1787827241.507148:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000001bd51b7e 17ffffc0001001 3:0 ffff9a78835e42b8 105 lru 00000080:00200000:2.0:1787827241.507154:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000141b9ad4 105: 1< 0 - - > 2< 430080 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507157:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006510a877 00000080:00200000:2.0:1787827241.507158:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 106. 00000080:00200000:2.0:1787827241.507160:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 106 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507174:0:48388:0:(rw26.c:875:ll_write_end()) pos 434176, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507179:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000fe56a86[3 00000000f7922827 2 1 000000000104970a 0 0 106] 00000080:00200000:2.0:1787827241.507182:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000d19b017 17ffffc0001001 3:0 ffff9a78835e41d0 106 lru 00000080:00200000:2.0:1787827241.507189:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002fe3ea59 106: 1< 0 - - > 2< 434176 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507191:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000fe56a86 00000080:00200000:2.0:1787827241.507193:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 107. 00000080:00200000:2.0:1787827241.507194:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 107 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507206:0:48388:0:(rw26.c:875:ll_write_end()) pos 438272, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507209:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005e60b5dc[3 00000000f7922827 2 1 000000000104970a 0 0 107] 00000080:00200000:2.0:1787827241.507211:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000076d9d197 17ffffc0001001 3:0 ffff9a78835e4740 107 lru 00000080:00200000:2.0:1787827241.507215:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000005e2e6f79 107: 1< 0 - - > 2< 438272 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507217:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005e60b5dc 00000080:00200000:2.0:1787827241.507218:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 108. 00000080:00200000:2.0:1787827241.507219:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 108 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507228:0:48388:0:(rw26.c:875:ll_write_end()) pos 442368, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507231:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ebd0504c[3 00000000f7922827 2 1 000000000104970a 0 0 108] 00000080:00200000:2.0:1787827241.507233:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000003d8bda77 17ffffc0001001 3:0 ffff9a78835e4e80 108 lru 00000080:00200000:2.0:1787827241.507237:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000022d9d72e 108: 1< 0 - - > 2< 442368 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507239:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ebd0504c 00000080:00200000:2.0:1787827241.507239:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 109. 00000080:00200000:2.0:1787827241.507240:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 109 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507248:0:48388:0:(rw26.c:875:ll_write_end()) pos 446464, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507251:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000094bddd17[3 00000000f7922827 2 1 000000000104970a 0 0 109] 00000080:00200000:2.0:1787827241.507252:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000024989b5d 17ffffc0001001 3:0 ffff9a78835e4488 109 lru 00000080:00200000:2.0:1787827241.507256:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000988896ba 109: 1< 0 - - > 2< 446464 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507257:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000094bddd17 00000080:00200000:2.0:1787827241.507258:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 110. 00000080:00200000:2.0:1787827241.507259:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 110 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507267:0:48388:0:(rw26.c:875:ll_write_end()) pos 450560, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507270:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000000c486bb[3 00000000f7922827 2 1 000000000104970a 0 0 110] 00000080:00200000:2.0:1787827241.507273:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000055eb2d2a 17ffffc0001001 3:0 ffff9a78835e4bc8 110 lru 00000080:00200000:2.0:1787827241.507279:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009fa43d51 110: 1< 0 - - > 2< 450560 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507282:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000000c486bb 00000080:00200000:2.0:1787827241.507283:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 111. 00000080:00200000:2.0:1787827241.507285:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 111 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507303:0:48388:0:(rw26.c:875:ll_write_end()) pos 454656, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507308:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ed76b42a[3 00000000f7922827 2 1 000000000104970a 0 0 111] 00000080:00200000:2.0:1787827241.507311:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000d7a8d28 17ffffc0001001 3:0 ffff9a78835e4570 111 lru 00000080:00200000:2.0:1787827241.507317:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000786179b4 111: 1< 0 - - > 2< 454656 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507320:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ed76b42a 00000080:00200000:2.0:1787827241.507321:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 112. 00000080:00200000:2.0:1787827241.507323:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 112 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507338:0:48388:0:(rw26.c:875:ll_write_end()) pos 458752, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507343:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000067280843[3 00000000f7922827 2 1 000000000104970a 0 0 112] 00000080:00200000:2.0:1787827241.507346:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e8679285 17ffffc0001001 3:0 ffff9a78835e4828 112 lru 00000080:00200000:2.0:1787827241.507353:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000f7a25c34 112: 1< 0 - - > 2< 458752 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507356:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000067280843 00000080:00200000:2.0:1787827241.507357:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 113. 00000080:00200000:2.0:1787827241.507359:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 113 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507376:0:48388:0:(rw26.c:875:ll_write_end()) pos 462848, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507381:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000a88c184[3 00000000f7922827 2 1 000000000104970a 0 0 113] 00000080:00200000:2.0:1787827241.507385:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c27b9b4f 17ffffc0001001 3:0 ffff9a78835e4000 113 lru 00000080:00200000:2.0:1787827241.507391:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007b5d4dc5 113: 1< 0 - - > 2< 462848 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507393:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000a88c184 00000080:00200000:2.0:1787827241.507395:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 114. 00000080:00200000:2.0:1787827241.507396:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 114 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507417:0:48388:0:(rw26.c:875:ll_write_end()) pos 466944, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507422:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d7db6290[3 00000000f7922827 2 1 000000000104970a 0 0 114] 00000080:00200000:2.0:1787827241.507426:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000022211a01 17ffffc0001021 2:0 ffff9a78835e4cb0 114 lru 00000080:00200000:2.0:1787827241.507432:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000021eefdea 114: 1< 0 - - > 2< 466944 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507435:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d7db6290 00000080:00200000:2.0:1787827241.507436:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 115. 00000080:00200000:2.0:1787827241.507438:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 115 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507454:0:48388:0:(rw26.c:875:ll_write_end()) pos 471040, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507459:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000024d071cb[3 00000000f7922827 2 1 000000000104970a 0 0 115] 00000080:00200000:2.0:1787827241.507462:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000018960c9f 17ffffc0001001 3:0 ffff9a78835e4ae0 115 lru 00000080:00200000:2.0:1787827241.507469:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a73d116f 115: 1< 0 - - > 2< 471040 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507471:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000024d071cb 00000080:00200000:2.0:1787827241.507473:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 116. 00000080:00200000:2.0:1787827241.507475:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 116 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507489:0:48388:0:(rw26.c:875:ll_write_end()) pos 475136, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507494:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000b9c881cd[3 00000000f7922827 2 1 000000000104970a 0 0 116] 00000080:00200000:2.0:1787827241.507497:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000066a53d18 17ffffc0001001 3:0 ffff9a78835e4658 116 lru 00000080:00200000:2.0:1787827241.507517:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000caa405fa 116: 1< 0 - - > 2< 475136 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507520:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000b9c881cd 00000080:00200000:2.0:1787827241.507522:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 117. 00000080:00200000:2.0:1787827241.507525:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 117 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507550:0:48388:0:(rw26.c:875:ll_write_end()) pos 479232, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507555:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000fde07d3c[3 00000000f7922827 2 1 000000000104970a 0 0 117] 00000080:00200000:2.0:1787827241.507558:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000cbfce4ae 17ffffc0001001 3:0 ffff9a7886cc1828 117 lru 00000080:00200000:2.0:1787827241.507564:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a8656a4f 117: 1< 0 - - > 2< 479232 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507567:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000fde07d3c 00000080:00200000:2.0:1787827241.507568:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 118. 00000080:00200000:2.0:1787827241.507570:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 118 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507590:0:48388:0:(rw26.c:875:ll_write_end()) pos 483328, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507596:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000036932292[3 00000000f7922827 2 1 000000000104970a 0 0 118] 00000080:00200000:2.0:1787827241.507599:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f484f11d 17ffffc0001001 3:0 ffff9a7886cc1000 118 lru 00000080:00200000:2.0:1787827241.507605:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d11c10ce 118: 1< 0 - - > 2< 483328 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507608:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000036932292 00000080:00200000:2.0:1787827241.507609:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 119. 00000080:00200000:2.0:1787827241.507611:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 119 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507626:0:48388:0:(rw26.c:875:ll_write_end()) pos 487424, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507632:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ad596729[3 00000000f7922827 2 1 000000000104970a 0 0 119] 00000080:00200000:2.0:1787827241.507635:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e10f9c37 17ffffc0001001 3:0 ffff9a7886cc1cb0 119 lru 00000080:00200000:2.0:1787827241.507640:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000022b4fcb 119: 1< 0 - - > 2< 487424 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507641:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ad596729 00000080:00200000:2.0:1787827241.507642:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 120. 00000080:00200000:2.0:1787827241.507644:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 120 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507655:0:48388:0:(rw26.c:875:ll_write_end()) pos 491520, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507658:0:48388:0:(rw26.c:933:ll_write_end()) page@000000004a2578a3[3 00000000f7922827 2 1 000000000104970a 0 0 120] 00000080:00200000:2.0:1787827241.507660:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d8ecd2a6 17ffffc0001001 3:0 ffff9a7886cc1ae0 120 lru 00000080:00200000:2.0:1787827241.507664:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000052a79e60 120: 1< 0 - - > 2< 491520 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507665:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000004a2578a3 00000080:00200000:2.0:1787827241.507666:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 121. 00000080:00200000:2.0:1787827241.507667:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 121 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507676:0:48388:0:(rw26.c:875:ll_write_end()) pos 495616, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507678:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000079d521cd[3 00000000f7922827 2 1 000000000104970a 0 0 121] 00000080:00200000:2.0:1787827241.507680:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000008017dd58 17ffffc0001001 3:0 ffff9a7886cc1658 121 lru 00000080:00200000:2.0:1787827241.507683:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000020d8b00e 121: 1< 0 - - > 2< 495616 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507686:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000079d521cd 00000080:00200000:2.0:1787827241.507687:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 122. 00000080:00200000:2.0:1787827241.507688:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 122 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507696:0:48388:0:(rw26.c:875:ll_write_end()) pos 499712, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507699:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000fb3c0303[3 00000000f7922827 2 1 000000000104970a 0 0 122] 00000080:00200000:2.0:1787827241.507701:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000038cff0da 17ffffc0001001 3:0 ffff9a7886cc19f8 122 lru 00000080:00200000:2.0:1787827241.507705:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a485b484 122: 1< 0 - - > 2< 499712 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507706:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000fb3c0303 00000080:00200000:2.0:1787827241.507707:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 123. 00000080:00200000:2.0:1787827241.507708:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 123 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507716:0:48388:0:(rw26.c:875:ll_write_end()) pos 503808, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507718:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e42f47f6[3 00000000f7922827 2 1 000000000104970a 0 0 123] 00000080:00200000:2.0:1787827241.507720:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000dddd0f88 17ffffc0001001 3:0 ffff9a7886cc1910 123 lru 00000080:00200000:2.0:1787827241.507723:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000012281205 123: 1< 0 - - > 2< 503808 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507724:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e42f47f6 00000080:00200000:2.0:1787827241.507725:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 124. 00000080:00200000:2.0:1787827241.507726:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 124 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507735:0:48388:0:(rw26.c:875:ll_write_end()) pos 507904, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507737:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000068a01651[3 00000000f7922827 2 1 000000000104970a 0 0 124] 00000080:00200000:2.0:1787827241.507739:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000008f139e6 17ffffc0001001 3:0 ffff9a7886cc1d98 124 lru 00000080:00200000:2.0:1787827241.507742:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e99a55c3 124: 1< 0 - - > 2< 507904 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507743:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000068a01651 00000080:00200000:2.0:1787827241.507744:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 125. 00000080:00200000:2.0:1787827241.507745:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 125 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507755:0:48388:0:(rw26.c:875:ll_write_end()) pos 512000, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507758:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a3e2a67e[3 00000000f7922827 2 1 000000000104970a 0 0 125] 00000080:00200000:2.0:1787827241.507759:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000cccaa8e7 17ffffc0001001 3:0 ffff9a7886cc13a0 125 lru 00000080:00200000:2.0:1787827241.507763:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000010b235ae 125: 1< 0 - - > 2< 512000 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507764:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a3e2a67e 00000080:00200000:2.0:1787827241.507765:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 126. 00000080:00200000:2.0:1787827241.507766:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 126 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507775:0:48388:0:(rw26.c:875:ll_write_end()) pos 516096, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507778:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000f4deb14a[3 00000000f7922827 2 1 000000000104970a 0 0 126] 00000080:00200000:2.0:1787827241.507780:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000004108df34 17ffffc0001001 3:0 ffff9a7886cc10e8 126 lru 00000080:00200000:2.0:1787827241.507783:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000018737643 126: 1< 0 - - > 2< 516096 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507785:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000f4deb14a 00000080:00200000:2.0:1787827241.507785:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 127. 00000080:00200000:2.0:1787827241.507786:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 127 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507794:0:48388:0:(rw26.c:875:ll_write_end()) pos 520192, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507797:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e5e6efc5[3 00000000f7922827 2 1 000000000104970a 0 0 127] 00000080:00200000:2.0:1787827241.507798:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009ea3ee64 17ffffc0001001 3:0 ffff9a7886cc12b8 127 lru 00000080:00200000:2.0:1787827241.507802:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000048d1b737 127: 1< 0 - - > 2< 520192 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507803:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e5e6efc5 00000080:00200000:2.0:1787827241.507804:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 128. 00000080:00200000:2.0:1787827241.507805:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 128 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.507814:0:48388:0:(rw26.c:875:ll_write_end()) pos 524288, len 4096, copied 4096 00000080:00200000:2.0:1787827241.507817:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a66474dc[3 00000000f7922827 2 1 000000000104970a 0 0 128] 00000080:00200000:2.0:1787827241.507819:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c3165106 17ffffc0001001 3:0 ffff9a7886cc11d0 128 lru 00000080:00200000:2.0:1787827241.507823:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000016189cfb 128: 1< 0 - - > 2< 524288 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.507825:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a66474dc 00000080:00200000:2.0:1787827241.507825:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 129. 00000080:00200000:2.0:1787827241.507826:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 129 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508058:0:48388:0:(rw26.c:875:ll_write_end()) pos 528384, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508061:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003f5acf05[3 00000000f7922827 2 1 000000000104970a 0 0 129] 00000080:00200000:2.0:1787827241.508063:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d2c17e16 17ffffc0001021 2:0 ffff9a7886cc1740 129 lru 00000080:00200000:2.0:1787827241.508067:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000010f2371d 129: 1< 0 - - > 2< 528384 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508069:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003f5acf05 00000080:00200000:2.0:1787827241.508070:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 130. 00000080:00200000:2.0:1787827241.508071:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 130 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508080:0:48388:0:(rw26.c:875:ll_write_end()) pos 532480, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508082:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000173ab375[3 00000000f7922827 2 1 000000000104970a 0 0 130] 00000080:00200000:2.0:1787827241.508084:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000945615e0 17ffffc0001001 3:0 ffff9a7886cc1e80 130 lru 00000080:00200000:2.0:1787827241.508088:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000fdd9afdc 130: 1< 0 - - > 2< 532480 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508090:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000173ab375 00000080:00200000:2.0:1787827241.508091:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 131. 00000080:00200000:2.0:1787827241.508092:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 131 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508101:0:48388:0:(rw26.c:875:ll_write_end()) pos 536576, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508103:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000860ba75a[3 00000000f7922827 2 1 000000000104970a 0 0 131] 00000080:00200000:2.0:1787827241.508105:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b583a776 17ffffc0001001 3:0 ffff9a7886cc1488 131 lru 00000080:00200000:2.0:1787827241.508109:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000035342b0 131: 1< 0 - - > 2< 536576 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508111:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000860ba75a 00000080:00200000:2.0:1787827241.508111:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 132. 00000080:00200000:2.0:1787827241.508112:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 132 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508121:0:48388:0:(rw26.c:875:ll_write_end()) pos 540672, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508123:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007e21cd5e[3 00000000f7922827 2 1 000000000104970a 0 0 132] 00000080:00200000:2.0:1787827241.508125:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f330929b 17ffffc0001001 3:0 ffff9a7886cc1bc8 132 lru 00000080:00200000:2.0:1787827241.508128:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000ad4f7298 132: 1< 0 - - > 2< 540672 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508129:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007e21cd5e 00000080:00200000:2.0:1787827241.508130:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 133. 00000080:00200000:2.0:1787827241.508131:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 133 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508139:0:48388:0:(rw26.c:875:ll_write_end()) pos 544768, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508141:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007741af9b[3 00000000f7922827 2 1 000000000104970a 0 0 133] 00000080:00200000:2.0:1787827241.508143:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c6bd994b 17ffffc0001001 3:0 ffff9a7886cc1570 133 lru 00000080:00200000:2.0:1787827241.508147:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000048a070c8 133: 1< 0 - - > 2< 544768 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508148:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007741af9b 00000080:00200000:2.0:1787827241.508148:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 134. 00000080:00200000:2.0:1787827241.508150:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 134 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508164:0:48388:0:(rw26.c:875:ll_write_end()) pos 548864, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508166:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a8ff2cdb[3 00000000f7922827 2 1 000000000104970a 0 0 134] 00000080:00200000:2.0:1787827241.508168:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000009d32e0b 17ffffc0001001 3:0 ffff9a78902a4570 134 lru 00000080:00200000:2.0:1787827241.508172:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000039e8b587 134: 1< 0 - - > 2< 548864 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508173:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a8ff2cdb 00000080:00200000:2.0:1787827241.508173:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 135. 00000080:00200000:2.0:1787827241.508174:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 135 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508184:0:48388:0:(rw26.c:875:ll_write_end()) pos 552960, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508186:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000b9147833[3 00000000f7922827 2 1 000000000104970a 0 0 135] 00000080:00200000:2.0:1787827241.508188:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b43325a8 17ffffc0001001 3:0 ffff9a78902a4828 135 lru 00000080:00200000:2.0:1787827241.508192:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003b796a3c 135: 1< 0 - - > 2< 552960 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508194:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000b9147833 00000080:00200000:2.0:1787827241.508194:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 136. 00000080:00200000:2.0:1787827241.508195:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 136 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508205:0:48388:0:(rw26.c:875:ll_write_end()) pos 557056, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508207:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000b184ae89[3 00000000f7922827 2 1 000000000104970a 0 0 136] 00000080:00200000:2.0:1787827241.508209:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002a6b1f02 17ffffc0001001 3:0 ffff9a78902a4000 136 lru 00000080:00200000:2.0:1787827241.508213:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000091244fbf 136: 1< 0 - - > 2< 557056 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508214:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000b184ae89 00000080:00200000:2.0:1787827241.508215:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 137. 00000080:00200000:2.0:1787827241.508216:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 137 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508225:0:48388:0:(rw26.c:875:ll_write_end()) pos 561152, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508228:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000053ecbfe6[3 00000000f7922827 2 1 000000000104970a 0 0 137] 00000080:00200000:2.0:1787827241.508230:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002e93c65a 17ffffc0001001 3:0 ffff9a78902a4cb0 137 lru 00000080:00200000:2.0:1787827241.508233:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c2b9efaa 137: 1< 0 - - > 2< 561152 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508234:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000053ecbfe6 00000080:00200000:2.0:1787827241.508235:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 138. 00000080:00200000:2.0:1787827241.508236:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 138 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508244:0:48388:0:(rw26.c:875:ll_write_end()) pos 565248, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508246:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000b6904959[3 00000000f7922827 2 1 000000000104970a 0 0 138] 00000080:00200000:2.0:1787827241.508248:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000059cc5a06 17ffffc0001001 3:0 ffff9a78902a4ae0 138 lru 00000080:00200000:2.0:1787827241.508252:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000094763d9a 138: 1< 0 - - > 2< 565248 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508253:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000b6904959 00000080:00200000:2.0:1787827241.508253:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 139. 00000080:00200000:2.0:1787827241.508254:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 139 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508263:0:48388:0:(rw26.c:875:ll_write_end()) pos 569344, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508265:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007ee17c22[3 00000000f7922827 2 1 000000000104970a 0 0 139] 00000080:00200000:2.0:1787827241.508267:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000023a5ca41 17ffffc0001001 3:0 ffff9a78902a4658 139 lru 00000080:00200000:2.0:1787827241.508270:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e1c0891c 139: 1< 0 - - > 2< 569344 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508273:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007ee17c22 00000080:00200000:2.0:1787827241.508273:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 140. 00000080:00200000:2.0:1787827241.508275:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 140 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508283:0:48388:0:(rw26.c:875:ll_write_end()) pos 573440, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508286:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000f3d46eba[3 00000000f7922827 2 1 000000000104970a 0 0 140] 00000080:00200000:2.0:1787827241.508288:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000073e8c78b 17ffffc0001001 3:0 ffff9a78902a49f8 140 lru 00000080:00200000:2.0:1787827241.508292:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003398294c 140: 1< 0 - - > 2< 573440 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508293:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000f3d46eba 00000080:00200000:2.0:1787827241.508294:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 141. 00000080:00200000:2.0:1787827241.508295:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 141 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508303:0:48388:0:(rw26.c:875:ll_write_end()) pos 577536, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508305:0:48388:0:(rw26.c:933:ll_write_end()) page@000000001035ae67[3 00000000f7922827 2 1 000000000104970a 0 0 141] 00000080:00200000:2.0:1787827241.508307:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ece518b6 17ffffc0001001 3:0 ffff9a78902a4910 141 lru 00000080:00200000:2.0:1787827241.508310:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000029d52907 141: 1< 0 - - > 2< 577536 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508311:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000001035ae67 00000080:00200000:2.0:1787827241.508312:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 142. 00000080:00200000:2.0:1787827241.508313:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 142 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508323:0:48388:0:(rw26.c:875:ll_write_end()) pos 581632, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508327:0:48388:0:(rw26.c:933:ll_write_end()) page@000000009953f834[3 00000000f7922827 2 1 000000000104970a 0 0 142] 00000080:00200000:2.0:1787827241.508330:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c2eaa515 17ffffc0001001 3:0 ffff9a78902a4d98 142 lru 00000080:00200000:2.0:1787827241.508336:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009a0c9656 142: 1< 0 - - > 2< 581632 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508338:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000009953f834 00000080:00200000:2.0:1787827241.508340:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 143. 00000080:00200000:2.0:1787827241.508342:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 143 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508359:0:48388:0:(rw26.c:875:ll_write_end()) pos 585728, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508364:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000aa3a74fe[3 00000000f7922827 2 1 000000000104970a 0 0 143] 00000080:00200000:2.0:1787827241.508367:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000bd22a4cd 17ffffc0001001 3:0 ffff9a78902a43a0 143 lru 00000080:00200000:2.0:1787827241.508374:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e2602182 143: 1< 0 - - > 2< 585728 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508376:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000aa3a74fe 00000080:00200000:2.0:1787827241.508377:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 144. 00000080:00200000:2.0:1787827241.508379:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 144 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508396:0:48388:0:(rw26.c:875:ll_write_end()) pos 589824, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508400:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000cc37d7a5[3 00000000f7922827 2 1 000000000104970a 0 0 144] 00000080:00200000:2.0:1787827241.508402:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000a81225be 17ffffc0001021 2:0 ffff9a78902a40e8 144 lru 00000080:00200000:2.0:1787827241.508406:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000af49ea50 144: 1< 0 - - > 2< 589824 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508407:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000cc37d7a5 00000080:00200000:2.0:1787827241.508408:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 145. 00000080:00200000:2.0:1787827241.508410:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 145 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508418:0:48388:0:(rw26.c:875:ll_write_end()) pos 593920, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508421:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000044208d73[3 00000000f7922827 2 1 000000000104970a 0 0 145] 00000080:00200000:2.0:1787827241.508423:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000139726b7 17ffffc0001001 3:0 ffff9a78902a42b8 145 lru 00000080:00200000:2.0:1787827241.508427:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000137d9d27 145: 1< 0 - - > 2< 593920 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508428:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000044208d73 00000080:00200000:2.0:1787827241.508429:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 146. 00000080:00200000:2.0:1787827241.508430:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 146 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508438:0:48388:0:(rw26.c:875:ll_write_end()) pos 598016, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508440:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003d2d6e6a[3 00000000f7922827 2 1 000000000104970a 0 0 146] 00000080:00200000:2.0:1787827241.508442:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005027a845 17ffffc0001001 3:0 ffff9a78902a41d0 146 lru 00000080:00200000:2.0:1787827241.508445:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000753cec8a 146: 1< 0 - - > 2< 598016 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508447:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003d2d6e6a 00000080:00200000:2.0:1787827241.508448:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 147. 00000080:00200000:2.0:1787827241.508449:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 147 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508457:0:48388:0:(rw26.c:875:ll_write_end()) pos 602112, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508460:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000bef4c988[3 00000000f7922827 2 1 000000000104970a 0 0 147] 00000080:00200000:2.0:1787827241.508462:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000fb9e3f5e 17ffffc0001001 3:0 ffff9a78902a4740 147 lru 00000080:00200000:2.0:1787827241.508465:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000cbc2183 147: 1< 0 - - > 2< 602112 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508467:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000bef4c988 00000080:00200000:2.0:1787827241.508468:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 148. 00000080:00200000:2.0:1787827241.508469:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 148 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508478:0:48388:0:(rw26.c:875:ll_write_end()) pos 606208, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508480:0:48388:0:(rw26.c:933:ll_write_end()) page@000000001ec99db4[3 00000000f7922827 2 1 000000000104970a 0 0 148] 00000080:00200000:2.0:1787827241.508482:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f9dd3db0 17ffffc0001001 3:0 ffff9a78902a4e80 148 lru 00000080:00200000:2.0:1787827241.508485:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000b0d80197 148: 1< 0 - - > 2< 606208 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508487:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000001ec99db4 00000080:00200000:2.0:1787827241.508488:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 149. 00000080:00200000:2.0:1787827241.508489:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 149 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508497:0:48388:0:(rw26.c:875:ll_write_end()) pos 610304, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508499:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ec7e4ee4[3 00000000f7922827 2 1 000000000104970a 0 0 149] 00000080:00200000:2.0:1787827241.508514:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000009aa1f5e 17ffffc0001001 3:0 ffff9a78902a4488 149 lru 00000080:00200000:2.0:1787827241.508520:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009f137519 149: 1< 0 - - > 2< 610304 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508521:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ec7e4ee4 00000080:00200000:2.0:1787827241.508522:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 150. 00000080:00200000:2.0:1787827241.508524:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 150 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508537:0:48388:0:(rw26.c:875:ll_write_end()) pos 614400, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508540:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000079100185[3 00000000f7922827 2 1 000000000104970a 0 0 150] 00000080:00200000:2.0:1787827241.508541:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000022b5999e 17ffffc0001001 3:0 ffff9a78902a4bc8 150 lru 00000080:00200000:2.0:1787827241.508545:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000eb636b65 150: 1< 0 - - > 2< 614400 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508547:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000079100185 00000080:00200000:2.0:1787827241.508548:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 151. 00000080:00200000:2.0:1787827241.508549:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 151 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508563:0:48388:0:(rw26.c:875:ll_write_end()) pos 618496, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508566:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ffc2cfd2[3 00000000f7922827 2 1 000000000104970a 0 0 151] 00000080:00200000:2.0:1787827241.508568:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000082905a6f 17ffffc0001001 3:0 ffff9a78a661c910 151 lru 00000080:00200000:2.0:1787827241.508571:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007aecaba6 151: 1< 0 - - > 2< 618496 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508573:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ffc2cfd2 00000080:00200000:2.0:1787827241.508574:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 152. 00000080:00200000:2.0:1787827241.508575:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 152 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508584:0:48388:0:(rw26.c:875:ll_write_end()) pos 622592, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508586:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d80f8653[3 00000000f7922827 2 1 000000000104970a 0 0 152] 00000080:00200000:2.0:1787827241.508588:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009319701c 17ffffc0001001 3:0 ffff9a78a661cd98 152 lru 00000080:00200000:2.0:1787827241.508592:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000008cc8ba4a 152: 1< 0 - - > 2< 622592 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508594:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d80f8653 00000080:00200000:2.0:1787827241.508595:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 153. 00000080:00200000:2.0:1787827241.508596:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 153 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508605:0:48388:0:(rw26.c:875:ll_write_end()) pos 626688, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508608:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006f3624df[3 00000000f7922827 2 1 000000000104970a 0 0 153] 00000080:00200000:2.0:1787827241.508610:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000078707376 17ffffc0001001 3:0 ffff9a78a661c3a0 153 lru 00000080:00200000:2.0:1787827241.508614:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a36f57e0 153: 1< 0 - - > 2< 626688 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508615:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006f3624df 00000080:00200000:2.0:1787827241.508615:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 154. 00000080:00200000:2.0:1787827241.508617:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 154 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508625:0:48388:0:(rw26.c:875:ll_write_end()) pos 630784, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508627:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006f9341aa[3 00000000f7922827 2 1 000000000104970a 0 0 154] 00000080:00200000:2.0:1787827241.508629:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002bc9f348 17ffffc0001001 3:0 ffff9a78a661c0e8 154 lru 00000080:00200000:2.0:1787827241.508632:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000700f249d 154: 1< 0 - - > 2< 630784 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508633:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006f9341aa 00000080:00200000:2.0:1787827241.508634:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 155. 00000080:00200000:2.0:1787827241.508635:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 155 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508643:0:48388:0:(rw26.c:875:ll_write_end()) pos 634880, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508646:0:48388:0:(rw26.c:933:ll_write_end()) page@000000008e627393[3 00000000f7922827 2 1 000000000104970a 0 0 155] 00000080:00200000:2.0:1787827241.508648:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000873217e2 17ffffc0001001 3:0 ffff9a78a661c2b8 155 lru 00000080:00200000:2.0:1787827241.508651:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000063a34e51 155: 1< 0 - - > 2< 634880 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508653:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000008e627393 00000080:00200000:2.0:1787827241.508654:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 156. 00000080:00200000:2.0:1787827241.508655:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 156 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508663:0:48388:0:(rw26.c:875:ll_write_end()) pos 638976, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508666:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000081b5c701[3 00000000f7922827 2 1 000000000104970a 0 0 156] 00000080:00200000:2.0:1787827241.508668:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000104aa00e 17ffffc0001001 3:0 ffff9a78a661c1d0 156 lru 00000080:00200000:2.0:1787827241.508671:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c00d4442 156: 1< 0 - - > 2< 638976 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508672:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000081b5c701 00000080:00200000:2.0:1787827241.508673:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 157. 00000080:00200000:2.0:1787827241.508674:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 157 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508684:0:48388:0:(rw26.c:875:ll_write_end()) pos 643072, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508686:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ec71e4f0[3 00000000f7922827 2 1 000000000104970a 0 0 157] 00000080:00200000:2.0:1787827241.508688:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005bafa543 17ffffc0001001 3:0 ffff9a78a661c740 157 lru 00000080:00200000:2.0:1787827241.508691:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000004507f7ca 157: 1< 0 - - > 2< 643072 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508693:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ec71e4f0 00000080:00200000:2.0:1787827241.508694:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 158. 00000080:00200000:2.0:1787827241.508695:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 158 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508703:0:48388:0:(rw26.c:875:ll_write_end()) pos 647168, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508706:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006b83fdf3[3 00000000f7922827 2 1 000000000104970a 0 0 158] 00000080:00200000:2.0:1787827241.508707:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000001362319 17ffffc0001001 3:0 ffff9a78a661ce80 158 lru 00000080:00200000:2.0:1787827241.508711:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000ae4341e2 158: 1< 0 - - > 2< 647168 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508712:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006b83fdf3 00000080:00200000:2.0:1787827241.508713:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 159. 00000080:00200000:2.0:1787827241.508714:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 159 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508724:0:48388:0:(rw26.c:875:ll_write_end()) pos 651264, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508727:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000033d1c9f0[3 00000000f7922827 2 1 000000000104970a 0 0 159] 00000080:00200000:2.0:1787827241.508729:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e8978127 17ffffc0001021 2:0 ffff9a78a661c488 159 lru 00000080:00200000:2.0:1787827241.508732:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002dee925d 159: 1< 0 - - > 2< 651264 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508734:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000033d1c9f0 00000080:00200000:2.0:1787827241.508735:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 160. 00000080:00200000:2.0:1787827241.508736:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 160 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508746:0:48388:0:(rw26.c:875:ll_write_end()) pos 655360, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508748:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000032f20f38[3 00000000f7922827 2 1 000000000104970a 0 0 160] 00000080:00200000:2.0:1787827241.508750:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000a2a8d83f 17ffffc0001001 3:0 ffff9a78a661cbc8 160 lru 00000080:00200000:2.0:1787827241.508754:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000b82cdfdf 160: 1< 0 - - > 2< 655360 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508756:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000032f20f38 00000080:00200000:2.0:1787827241.508757:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 161. 00000080:00200000:2.0:1787827241.508758:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 161 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508766:0:48388:0:(rw26.c:875:ll_write_end()) pos 659456, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508768:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e810f8be[3 00000000f7922827 2 1 000000000104970a 0 0 161] 00000080:00200000:2.0:1787827241.508770:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ff35b390 17ffffc0001001 3:0 ffff9a78a661c570 161 lru 00000080:00200000:2.0:1787827241.508773:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000b75e7970 161: 1< 0 - - > 2< 659456 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508775:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e810f8be 00000080:00200000:2.0:1787827241.508775:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 162. 00000080:00200000:2.0:1787827241.508776:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 162 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508785:0:48388:0:(rw26.c:875:ll_write_end()) pos 663552, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508789:0:48388:0:(rw26.c:933:ll_write_end()) page@000000002914f0e2[3 00000000f7922827 2 1 000000000104970a 0 0 162] 00000080:00200000:2.0:1787827241.508791:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007bbc5b4b 17ffffc0001001 3:0 ffff9a78a661c828 162 lru 00000080:00200000:2.0:1787827241.508797:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006fe64502 162: 1< 0 - - > 2< 663552 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508799:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000002914f0e2 00000080:00200000:2.0:1787827241.508800:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 163. 00000080:00200000:2.0:1787827241.508802:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 163 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508817:0:48388:0:(rw26.c:875:ll_write_end()) pos 667648, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508822:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000019afcdb1[3 00000000f7922827 2 1 000000000104970a 0 0 163] 00000080:00200000:2.0:1787827241.508825:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e4d81b7f 17ffffc0001001 3:0 ffff9a78a661c000 163 lru 00000080:00200000:2.0:1787827241.508847:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e305bd46 163: 1< 0 - - > 2< 667648 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508849:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000019afcdb1 00000080:00200000:2.0:1787827241.508850:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 164. 00000080:00200000:2.0:1787827241.508852:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 164 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508866:0:48388:0:(rw26.c:875:ll_write_end()) pos 671744, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508869:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003ac18872[3 00000000f7922827 2 1 000000000104970a 0 0 164] 00000080:00200000:2.0:1787827241.508871:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000042951d40 17ffffc0001001 3:0 ffff9a78a661ccb0 164 lru 00000080:00200000:2.0:1787827241.508875:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000004207a182 164: 1< 0 - - > 2< 671744 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508877:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003ac18872 00000080:00200000:2.0:1787827241.508878:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 165. 00000080:00200000:2.0:1787827241.508879:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 165 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508888:0:48388:0:(rw26.c:875:ll_write_end()) pos 675840, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508891:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000013b84850[3 00000000f7922827 2 1 000000000104970a 0 0 165] 00000080:00200000:2.0:1787827241.508893:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000859899b6 17ffffc0001001 3:0 ffff9a78a661cae0 165 lru 00000080:00200000:2.0:1787827241.508897:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000bb77d927 165: 1< 0 - - > 2< 675840 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508898:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000013b84850 00000080:00200000:2.0:1787827241.508899:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 166. 00000080:00200000:2.0:1787827241.508900:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 166 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508911:0:48388:0:(rw26.c:875:ll_write_end()) pos 679936, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508916:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000069ffc415[3 00000000f7922827 2 1 000000000104970a 0 0 166] 00000080:00200000:2.0:1787827241.508918:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000fc569c35 17ffffc0001001 3:0 ffff9a78a661c658 166 lru 00000080:00200000:2.0:1787827241.508924:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000bfc1587d 166: 1< 0 - - > 2< 679936 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508927:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000069ffc415 00000080:00200000:2.0:1787827241.508928:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 167. 00000080:00200000:2.0:1787827241.508931:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 167 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508946:0:48388:0:(rw26.c:875:ll_write_end()) pos 684032, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508950:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000099d7dece[3 00000000f7922827 2 1 000000000104970a 0 0 167] 00000080:00200000:2.0:1787827241.508953:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000aaf36151 17ffffc0001001 3:0 ffff9a78a661c9f8 167 lru 00000080:00200000:2.0:1787827241.508960:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000b61a80a2 167: 1< 0 - - > 2< 684032 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.508962:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000099d7dece 00000080:00200000:2.0:1787827241.508964:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 168. 00000080:00200000:2.0:1787827241.508966:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 168 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.508988:0:48388:0:(rw26.c:875:ll_write_end()) pos 688128, len 4096, copied 4096 00000080:00200000:2.0:1787827241.508994:0:48388:0:(rw26.c:933:ll_write_end()) page@000000009691e927[3 00000000f7922827 2 1 000000000104970a 0 0 168] 00000080:00200000:2.0:1787827241.508998:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000039952985 17ffffc0001001 3:0 ffff9a78909a2e80 168 lru 00000080:00200000:2.0:1787827241.509004:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000036fb2121 168: 1< 0 - - > 2< 688128 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509006:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000009691e927 00000080:00200000:2.0:1787827241.509008:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 169. 00000080:00200000:2.0:1787827241.509010:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 169 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509027:0:48388:0:(rw26.c:875:ll_write_end()) pos 692224, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509033:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e32a5c32[3 00000000f7922827 2 1 000000000104970a 0 0 169] 00000080:00200000:2.0:1787827241.509036:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007fb4b3c6 17ffffc0001001 3:0 ffff9a78909a2488 169 lru 00000080:00200000:2.0:1787827241.509042:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000060b815ac 169: 1< 0 - - > 2< 692224 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509044:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e32a5c32 00000080:00200000:2.0:1787827241.509046:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 170. 00000080:00200000:2.0:1787827241.509048:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 170 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509065:0:48388:0:(rw26.c:875:ll_write_end()) pos 696320, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509070:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d69d6fbd[3 00000000f7922827 2 1 000000000104970a 0 0 170] 00000080:00200000:2.0:1787827241.509073:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000f4572fd 17ffffc0001001 3:0 ffff9a78909a2bc8 170 lru 00000080:00200000:2.0:1787827241.509080:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e40201e7 170: 1< 0 - - > 2< 696320 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509082:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d69d6fbd 00000080:00200000:2.0:1787827241.509084:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 171. 00000080:00200000:2.0:1787827241.509086:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 171 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509103:0:48388:0:(rw26.c:875:ll_write_end()) pos 700416, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509109:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006c348eaf[3 00000000f7922827 2 1 000000000104970a 0 0 171] 00000080:00200000:2.0:1787827241.509112:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000dc39f4c6 17ffffc0001001 3:0 ffff9a78909a2570 171 lru 00000080:00200000:2.0:1787827241.509118:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a31cfa39 171: 1< 0 - - > 2< 700416 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509121:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006c348eaf 00000080:00200000:2.0:1787827241.509122:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 172. 00000080:00200000:2.0:1787827241.509124:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 172 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509138:0:48388:0:(rw26.c:875:ll_write_end()) pos 704512, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509143:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000d22b4823[3 00000000f7922827 2 1 000000000104970a 0 0 172] 00000080:00200000:2.0:1787827241.509147:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000268e54c8 17ffffc0001001 3:0 ffff9a78909a2828 172 lru 00000080:00200000:2.0:1787827241.509153:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a6ea6702 172: 1< 0 - - > 2< 704512 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509156:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000d22b4823 00000080:00200000:2.0:1787827241.509157:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 173. 00000080:00200000:2.0:1787827241.509159:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 173 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509173:0:48388:0:(rw26.c:875:ll_write_end()) pos 708608, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509178:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c634ab07[3 00000000f7922827 2 1 000000000104970a 0 0 173] 00000080:00200000:2.0:1787827241.509181:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000500641a0 17ffffc0001001 3:0 ffff9a78909a2000 173 lru 00000080:00200000:2.0:1787827241.509187:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d3b69ebd 173: 1< 0 - - > 2< 708608 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509190:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c634ab07 00000080:00200000:2.0:1787827241.509191:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 174. 00000080:00200000:2.0:1787827241.509193:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 174 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509214:0:48388:0:(rw26.c:875:ll_write_end()) pos 712704, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509219:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003a60362d[3 00000000f7922827 2 1 000000000104970a 0 0 174] 00000080:00200000:2.0:1787827241.509223:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ae07ce67 17ffffc0001021 2:0 ffff9a78909a2cb0 174 lru 00000080:00200000:2.0:1787827241.509229:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007918b70c 174: 1< 0 - - > 2< 712704 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509233:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003a60362d 00000080:00200000:2.0:1787827241.509234:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 175. 00000080:00200000:2.0:1787827241.509236:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 175 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509252:0:48388:0:(rw26.c:875:ll_write_end()) pos 716800, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509256:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000059c9b45e[3 00000000f7922827 2 1 000000000104970a 0 0 175] 00000080:00200000:2.0:1787827241.509258:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c81144f3 17ffffc0001001 3:0 ffff9a78909a2ae0 175 lru 00000080:00200000:2.0:1787827241.509262:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001a1c924d 175: 1< 0 - - > 2< 716800 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509264:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000059c9b45e 00000080:00200000:2.0:1787827241.509265:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 176. 00000080:00200000:2.0:1787827241.509266:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 176 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509276:0:48388:0:(rw26.c:875:ll_write_end()) pos 720896, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509279:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000067262410[3 00000000f7922827 2 1 000000000104970a 0 0 176] 00000080:00200000:2.0:1787827241.509281:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000ed12a43 17ffffc0001001 3:0 ffff9a78909a2658 176 lru 00000080:00200000:2.0:1787827241.509285:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e03b98e6 176: 1< 0 - - > 2< 720896 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509287:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000067262410 00000080:00200000:2.0:1787827241.509287:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 177. 00000080:00200000:2.0:1787827241.509288:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 177 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509296:0:48388:0:(rw26.c:875:ll_write_end()) pos 724992, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509299:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ba40ab68[3 00000000f7922827 2 1 000000000104970a 0 0 177] 00000080:00200000:2.0:1787827241.509301:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002dcb857b 17ffffc0001001 3:0 ffff9a78909a29f8 177 lru 00000080:00200000:2.0:1787827241.509304:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000091a51814 177: 1< 0 - - > 2< 724992 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509305:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ba40ab68 00000080:00200000:2.0:1787827241.509306:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 178. 00000080:00200000:2.0:1787827241.509307:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 178 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509317:0:48388:0:(rw26.c:875:ll_write_end()) pos 729088, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509321:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003058a7fe[3 00000000f7922827 2 1 000000000104970a 0 0 178] 00000080:00200000:2.0:1787827241.509324:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ff303b3f 17ffffc0001001 3:0 ffff9a78909a2910 178 lru 00000080:00200000:2.0:1787827241.509331:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000039435afd 178: 1< 0 - - > 2< 729088 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509333:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003058a7fe 00000080:00200000:2.0:1787827241.509335:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 179. 00000080:00200000:2.0:1787827241.509336:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 179 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509352:0:48388:0:(rw26.c:875:ll_write_end()) pos 733184, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509357:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000297fa3b3[3 00000000f7922827 2 1 000000000104970a 0 0 179] 00000080:00200000:2.0:1787827241.509360:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000998fd25d 17ffffc0001001 3:0 ffff9a78909a2d98 179 lru 00000080:00200000:2.0:1787827241.509365:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003f4cdf6d 179: 1< 0 - - > 2< 733184 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509368:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000297fa3b3 00000080:00200000:2.0:1787827241.509369:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 180. 00000080:00200000:2.0:1787827241.509371:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 180 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509386:0:48388:0:(rw26.c:875:ll_write_end()) pos 737280, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509390:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000322515d3[3 00000000f7922827 2 1 000000000104970a 0 0 180] 00000080:00200000:2.0:1787827241.509394:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000001fb3b4cd 17ffffc0001001 3:0 ffff9a78909a23a0 180 lru 00000080:00200000:2.0:1787827241.509399:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002c81cd60 180: 1< 0 - - > 2< 737280 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509401:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000322515d3 00000080:00200000:2.0:1787827241.509402:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 181. 00000080:00200000:2.0:1787827241.509404:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 181 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509416:0:48388:0:(rw26.c:875:ll_write_end()) pos 741376, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509419:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000023f4dfb8[3 00000000f7922827 2 1 000000000104970a 0 0 181] 00000080:00200000:2.0:1787827241.509420:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009dcfbecd 17ffffc0001001 3:0 ffff9a78909a20e8 181 lru 00000080:00200000:2.0:1787827241.509424:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002fc68167 181: 1< 0 - - > 2< 741376 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509425:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000023f4dfb8 00000080:00200000:2.0:1787827241.509426:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 182. 00000080:00200000:2.0:1787827241.509427:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 182 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509435:0:48388:0:(rw26.c:875:ll_write_end()) pos 745472, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509438:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000100b5e86[3 00000000f7922827 2 1 000000000104970a 0 0 182] 00000080:00200000:2.0:1787827241.509439:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ad31bdab 17ffffc0001001 3:0 ffff9a78909a22b8 182 lru 00000080:00200000:2.0:1787827241.509443:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000088ae48ce 182: 1< 0 - - > 2< 745472 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509444:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000100b5e86 00000080:00200000:2.0:1787827241.509444:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 183. 00000080:00200000:2.0:1787827241.509446:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 183 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509454:0:48388:0:(rw26.c:875:ll_write_end()) pos 749568, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509456:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000009ed93db[3 00000000f7922827 2 1 000000000104970a 0 0 183] 00000080:00200000:2.0:1787827241.509459:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000524bb5e2 17ffffc0001001 3:0 ffff9a78909a21d0 183 lru 00000080:00200000:2.0:1787827241.509462:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e7517f36 183: 1< 0 - - > 2< 749568 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509464:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000009ed93db 00000080:00200000:2.0:1787827241.509464:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 184. 00000080:00200000:2.0:1787827241.509465:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 184 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509474:0:48388:0:(rw26.c:875:ll_write_end()) pos 753664, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509476:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ed6bf1fe[3 00000000f7922827 2 1 000000000104970a 0 0 184] 00000080:00200000:2.0:1787827241.509478:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009ac0b131 17ffffc0001001 3:0 ffff9a78909a2740 184 lru 00000080:00200000:2.0:1787827241.509481:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000004b383e22 184: 1< 0 - - > 2< 753664 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509483:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ed6bf1fe 00000080:00200000:2.0:1787827241.509484:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 185. 00000080:00200000:2.0:1787827241.509485:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 185 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509516:0:48388:0:(rw26.c:875:ll_write_end()) pos 757760, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509520:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007d6a27f6[3 00000000f7922827 2 1 000000000104970a 0 0 185] 00000080:00200000:2.0:1787827241.509521:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000622a90dd 17ffffc0001001 3:0 ffff9a78812e5910 185 lru 00000080:00200000:2.0:1787827241.509525:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000735b9385 185: 1< 0 - - > 2< 757760 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509527:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007d6a27f6 00000080:00200000:2.0:1787827241.509528:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 186. 00000080:00200000:2.0:1787827241.509530:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 186 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509545:0:48388:0:(rw26.c:875:ll_write_end()) pos 761856, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509550:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000397269f4[3 00000000f7922827 2 1 000000000104970a 0 0 186] 00000080:00200000:2.0:1787827241.509553:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000afa3d061 17ffffc0001001 3:0 ffff9a78812e5d98 186 lru 00000080:00200000:2.0:1787827241.509560:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007e316a45 186: 1< 0 - - > 2< 761856 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509562:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000397269f4 00000080:00200000:2.0:1787827241.509564:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 187. 00000080:00200000:2.0:1787827241.509566:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 187 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509579:0:48388:0:(rw26.c:875:ll_write_end()) pos 765952, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509582:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000087cfd119[3 00000000f7922827 2 1 000000000104970a 0 0 187] 00000080:00200000:2.0:1787827241.509584:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000055462247 17ffffc0001001 3:0 ffff9a78812e53a0 187 lru 00000080:00200000:2.0:1787827241.509588:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000059416021 187: 1< 0 - - > 2< 765952 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509590:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000087cfd119 00000080:00200000:2.0:1787827241.509591:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 188. 00000080:00200000:2.0:1787827241.509592:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 188 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509601:0:48388:0:(rw26.c:875:ll_write_end()) pos 770048, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509603:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000522eed11[3 00000000f7922827 2 1 000000000104970a 0 0 188] 00000080:00200000:2.0:1787827241.509605:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000173038ca 17ffffc0001001 3:0 ffff9a78812e50e8 188 lru 00000080:00200000:2.0:1787827241.509609:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006d9ad58f 188: 1< 0 - - > 2< 770048 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509611:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000522eed11 00000080:00200000:2.0:1787827241.509612:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 189. 00000080:00200000:2.0:1787827241.509613:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 189 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509624:0:48388:0:(rw26.c:875:ll_write_end()) pos 774144, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509627:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000621560b6[3 00000000f7922827 2 1 000000000104970a 0 0 189] 00000080:00200000:2.0:1787827241.509629:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000008e5995cd 17ffffc0001021 2:0 ffff9a78812e52b8 189 lru 00000080:00200000:2.0:1787827241.509633:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000099fd54b9 189: 1< 0 - - > 2< 774144 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509635:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000621560b6 00000080:00200000:2.0:1787827241.509635:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 190. 00000080:00200000:2.0:1787827241.509636:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 190 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509645:0:48388:0:(rw26.c:875:ll_write_end()) pos 778240, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509647:0:48388:0:(rw26.c:933:ll_write_end()) page@000000009eb6e61a[3 00000000f7922827 2 1 000000000104970a 0 0 190] 00000080:00200000:2.0:1787827241.509649:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d1171b40 17ffffc0001001 3:0 ffff9a78812e51d0 190 lru 00000080:00200000:2.0:1787827241.509652:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003ecab982 190: 1< 0 - - > 2< 778240 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509654:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000009eb6e61a 00000080:00200000:2.0:1787827241.509654:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 191. 00000080:00200000:2.0:1787827241.509655:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 191 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509664:0:48388:0:(rw26.c:875:ll_write_end()) pos 782336, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509666:0:48388:0:(rw26.c:933:ll_write_end()) page@000000001814a6fe[3 00000000f7922827 2 1 000000000104970a 0 0 191] 00000080:00200000:2.0:1787827241.509668:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007bea8a7b 17ffffc0001001 3:0 ffff9a78812e5740 191 lru 00000080:00200000:2.0:1787827241.509672:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000cba4ddae 191: 1< 0 - - > 2< 782336 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509673:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000001814a6fe 00000080:00200000:2.0:1787827241.509674:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 192. 00000080:00200000:2.0:1787827241.509675:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 192 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509688:0:48388:0:(rw26.c:875:ll_write_end()) pos 786432, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509691:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003134e9b9[3 00000000f7922827 2 1 000000000104970a 0 0 192] 00000080:00200000:2.0:1787827241.509694:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d5773444 17ffffc0001001 3:0 ffff9a78812e5e80 192 lru 00000080:00200000:2.0:1787827241.509700:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000f609ca68 192: 1< 0 - - > 2< 786432 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509702:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003134e9b9 00000080:00200000:2.0:1787827241.509703:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 193. 00000080:00200000:2.0:1787827241.509705:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 193 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509721:0:48388:0:(rw26.c:875:ll_write_end()) pos 790528, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509726:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a9750079[3 00000000f7922827 2 1 000000000104970a 0 0 193] 00000080:00200000:2.0:1787827241.509729:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000ccaff8cc 17ffffc0001001 3:0 ffff9a78812e5488 193 lru 00000080:00200000:2.0:1787827241.509736:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003fe986f0 193: 1< 0 - - > 2< 790528 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509738:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a9750079 00000080:00200000:2.0:1787827241.509740:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 194. 00000080:00200000:2.0:1787827241.509742:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 194 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509758:0:48388:0:(rw26.c:875:ll_write_end()) pos 794624, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509764:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000091d7a931[3 00000000f7922827 2 1 000000000104970a 0 0 194] 00000080:00200000:2.0:1787827241.509767:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000229df08d 17ffffc0001001 3:0 ffff9a78812e5bc8 194 lru 00000080:00200000:2.0:1787827241.509774:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002142f577 194: 1< 0 - - > 2< 794624 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509776:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000091d7a931 00000080:00200000:2.0:1787827241.509777:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 195. 00000080:00200000:2.0:1787827241.509779:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 195 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509791:0:48388:0:(rw26.c:875:ll_write_end()) pos 798720, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509794:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000cb77186e[3 00000000f7922827 2 1 000000000104970a 0 0 195] 00000080:00200000:2.0:1787827241.509796:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c368e456 17ffffc0001001 3:0 ffff9a78812e5570 195 lru 00000080:00200000:2.0:1787827241.509799:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002abfa53a 195: 1< 0 - - > 2< 798720 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509801:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000cb77186e 00000080:00200000:2.0:1787827241.509802:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 196. 00000080:00200000:2.0:1787827241.509803:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 196 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509811:0:48388:0:(rw26.c:875:ll_write_end()) pos 802816, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509814:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e3c9f100[3 00000000f7922827 2 1 000000000104970a 0 0 196] 00000080:00200000:2.0:1787827241.509815:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000004321c9d5 17ffffc0001001 3:0 ffff9a78812e5828 196 lru 00000080:00200000:2.0:1787827241.509819:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d3aeb2ee 196: 1< 0 - - > 2< 802816 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509820:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e3c9f100 00000080:00200000:2.0:1787827241.509821:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 197. 00000080:00200000:2.0:1787827241.509822:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 197 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509847:0:48388:0:(rw26.c:875:ll_write_end()) pos 806912, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509850:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000263e91a1[3 00000000f7922827 2 1 000000000104970a 0 0 197] 00000080:00200000:2.0:1787827241.509852:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000240258a8 17ffffc0001001 3:0 ffff9a78812e5000 197 lru 00000080:00200000:2.0:1787827241.509857:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000000b25f971 197: 1< 0 - - > 2< 806912 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509858:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000263e91a1 00000080:00200000:2.0:1787827241.509859:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 198. 00000080:00200000:2.0:1787827241.509860:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 198 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509871:0:48388:0:(rw26.c:875:ll_write_end()) pos 811008, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509874:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c012e22e[3 00000000f7922827 2 1 000000000104970a 0 0 198] 00000080:00200000:2.0:1787827241.509876:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000067fd07e 17ffffc0001001 3:0 ffff9a78812e5cb0 198 lru 00000080:00200000:2.0:1787827241.509879:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000044e6385e 198: 1< 0 - - > 2< 811008 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509881:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c012e22e 00000080:00200000:2.0:1787827241.509882:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 199. 00000080:00200000:2.0:1787827241.509883:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 199 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509892:0:48388:0:(rw26.c:875:ll_write_end()) pos 815104, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509895:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000300fcb23[3 00000000f7922827 2 1 000000000104970a 0 0 199] 00000080:00200000:2.0:1787827241.509897:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b5ddafc6 17ffffc0001001 3:0 ffff9a78812e5ae0 199 lru 00000080:00200000:2.0:1787827241.509900:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000279e7fb3 199: 1< 0 - - > 2< 815104 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509902:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000300fcb23 00000080:00200000:2.0:1787827241.509902:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 200. 00000080:00200000:2.0:1787827241.509903:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 200 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509912:0:48388:0:(rw26.c:875:ll_write_end()) pos 819200, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509914:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ab4c0742[3 00000000f7922827 2 1 000000000104970a 0 0 200] 00000080:00200000:2.0:1787827241.509916:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d7a9b40b 17ffffc0001001 3:0 ffff9a78812e5658 200 lru 00000080:00200000:2.0:1787827241.509919:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000005afc7e8 200: 1< 0 - - > 2< 819200 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509920:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ab4c0742 00000080:00200000:2.0:1787827241.509921:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 201. 00000080:00200000:2.0:1787827241.509922:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 201 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509930:0:48388:0:(rw26.c:875:ll_write_end()) pos 823296, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509933:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e974b962[3 00000000f7922827 2 1 000000000104970a 0 0 201] 00000080:00200000:2.0:1787827241.509935:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000165ab14d 17ffffc0001001 3:0 ffff9a78812e59f8 201 lru 00000080:00200000:2.0:1787827241.509938:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000003c9abb49 201: 1< 0 - - > 2< 823296 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509939:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e974b962 00000080:00200000:2.0:1787827241.509940:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 202. 00000080:00200000:2.0:1787827241.509941:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 202 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509956:0:48388:0:(rw26.c:875:ll_write_end()) pos 827392, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509959:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000f42739ae[3 00000000f7922827 2 1 000000000104970a 0 0 202] 00000080:00200000:2.0:1787827241.509960:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000f5889498 17ffffc0001001 3:0 ffff9a78869542b8 202 lru 00000080:00200000:2.0:1787827241.509964:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007630ffc8 202: 1< 0 - - > 2< 827392 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509966:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000f42739ae 00000080:00200000:2.0:1787827241.509967:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 203. 00000080:00200000:2.0:1787827241.509968:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 203 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.509977:0:48388:0:(rw26.c:875:ll_write_end()) pos 831488, len 4096, copied 4096 00000080:00200000:2.0:1787827241.509980:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006676c296[3 00000000f7922827 2 1 000000000104970a 0 0 203] 00000080:00200000:2.0:1787827241.509982:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000037683a81 17ffffc0001001 3:0 ffff9a78869541d0 203 lru 00000080:00200000:2.0:1787827241.509986:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001da71690 203: 1< 0 - - > 2< 831488 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.509987:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006676c296 00000080:00200000:2.0:1787827241.509988:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 204. 00000080:00200000:2.0:1787827241.509989:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 204 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510000:0:48388:0:(rw26.c:875:ll_write_end()) pos 835584, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510002:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000294dd25[3 00000000f7922827 2 1 000000000104970a 0 0 204] 00000080:00200000:2.0:1787827241.510004:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000026d886c2 17ffffc0001021 2:0 ffff9a7886954740 204 lru 00000080:00200000:2.0:1787827241.510008:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001ed4217e 204: 1< 0 - - > 2< 835584 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510009:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000294dd25 00000080:00200000:2.0:1787827241.510010:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 205. 00000080:00200000:2.0:1787827241.510011:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 205 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510019:0:48388:0:(rw26.c:875:ll_write_end()) pos 839680, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510021:0:48388:0:(rw26.c:933:ll_write_end()) page@000000008b580f27[3 00000000f7922827 2 1 000000000104970a 0 0 205] 00000080:00200000:2.0:1787827241.510023:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000297c5afd 17ffffc0001001 3:0 ffff9a7886954e80 205 lru 00000080:00200000:2.0:1787827241.510027:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000324c602c 205: 1< 0 - - > 2< 839680 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510028:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000008b580f27 00000080:00200000:2.0:1787827241.510029:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 206. 00000080:00200000:2.0:1787827241.510030:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 206 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510040:0:48388:0:(rw26.c:875:ll_write_end()) pos 843776, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510042:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000067ef1433[3 00000000f7922827 2 1 000000000104970a 0 0 206] 00000080:00200000:2.0:1787827241.510044:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000057dfc932 17ffffc0001001 3:0 ffff9a7886954488 206 lru 00000080:00200000:2.0:1787827241.510048:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000fbeb7537 206: 1< 0 - - > 2< 843776 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510050:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000067ef1433 00000080:00200000:2.0:1787827241.510051:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 207. 00000080:00200000:2.0:1787827241.510052:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 207 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510061:0:48388:0:(rw26.c:875:ll_write_end()) pos 847872, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510064:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a7da6319[3 00000000f7922827 2 1 000000000104970a 0 0 207] 00000080:00200000:2.0:1787827241.510066:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002d726ae6 17ffffc0001001 3:0 ffff9a7886954bc8 207 lru 00000080:00200000:2.0:1787827241.510069:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000111250e5 207: 1< 0 - - > 2< 847872 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510071:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a7da6319 00000080:00200000:2.0:1787827241.510071:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 208. 00000080:00200000:2.0:1787827241.510072:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 208 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510081:0:48388:0:(rw26.c:875:ll_write_end()) pos 851968, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510083:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000439f17b5[3 00000000f7922827 2 1 000000000104970a 0 0 208] 00000080:00200000:2.0:1787827241.510085:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c44878e1 17ffffc0001001 3:0 ffff9a7886954570 208 lru 00000080:00200000:2.0:1787827241.510088:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000eec69899 208: 1< 0 - - > 2< 851968 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510089:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000439f17b5 00000080:00200000:2.0:1787827241.510090:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 209. 00000080:00200000:2.0:1787827241.510091:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 209 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510099:0:48388:0:(rw26.c:875:ll_write_end()) pos 856064, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510101:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000cd6b3b9b[3 00000000f7922827 2 1 000000000104970a 0 0 209] 00000080:00200000:2.0:1787827241.510103:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007cedd19c 17ffffc0001001 3:0 ffff9a7886954828 209 lru 00000080:00200000:2.0:1787827241.510106:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000049addf44 209: 1< 0 - - > 2< 856064 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510108:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000cd6b3b9b 00000080:00200000:2.0:1787827241.510108:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 210. 00000080:00200000:2.0:1787827241.510109:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 210 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510117:0:48388:0:(rw26.c:875:ll_write_end()) pos 860160, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510120:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005cf84a4e[3 00000000f7922827 2 1 000000000104970a 0 0 210] 00000080:00200000:2.0:1787827241.510122:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d431fa68 17ffffc0001001 3:0 ffff9a7886954000 210 lru 00000080:00200000:2.0:1787827241.510125:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009fc08f8d 210: 1< 0 - - > 2< 860160 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510126:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005cf84a4e 00000080:00200000:2.0:1787827241.510127:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 211. 00000080:00200000:2.0:1787827241.510128:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 211 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510136:0:48388:0:(rw26.c:875:ll_write_end()) pos 864256, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510138:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000003f135a9[3 00000000f7922827 2 1 000000000104970a 0 0 211] 00000080:00200000:2.0:1787827241.510140:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c5980c5e 17ffffc0001001 3:0 ffff9a7886954cb0 211 lru 00000080:00200000:2.0:1787827241.510144:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a8069561 211: 1< 0 - - > 2< 864256 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510146:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000003f135a9 00000080:00200000:2.0:1787827241.510146:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 212. 00000080:00200000:2.0:1787827241.510147:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 212 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510159:0:48388:0:(rw26.c:875:ll_write_end()) pos 868352, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510164:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000945c0f13[3 00000000f7922827 2 1 000000000104970a 0 0 212] 00000080:00200000:2.0:1787827241.510167:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000298adac1 17ffffc0001001 3:0 ffff9a7886954ae0 212 lru 00000080:00200000:2.0:1787827241.510172:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000005819dfd8 212: 1< 0 - - > 2< 868352 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510174:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000945c0f13 00000080:00200000:2.0:1787827241.510175:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 213. 00000080:00200000:2.0:1787827241.510177:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 213 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510193:0:48388:0:(rw26.c:875:ll_write_end()) pos 872448, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510197:0:48388:0:(rw26.c:933:ll_write_end()) page@000000002e83d541[3 00000000f7922827 2 1 000000000104970a 0 0 213] 00000080:00200000:2.0:1787827241.510199:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000002003d047 17ffffc0001001 3:0 ffff9a7886954658 213 lru 00000080:00200000:2.0:1787827241.510203:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c1e7d9ea 213: 1< 0 - - > 2< 872448 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510204:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000002e83d541 00000080:00200000:2.0:1787827241.510205:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 214. 00000080:00200000:2.0:1787827241.510207:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 214 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510217:0:48388:0:(rw26.c:875:ll_write_end()) pos 876544, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510222:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000f686f92b[3 00000000f7922827 2 1 000000000104970a 0 0 214] 00000080:00200000:2.0:1787827241.510225:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000004fe0d55f 17ffffc0001001 3:0 ffff9a78869549f8 214 lru 00000080:00200000:2.0:1787827241.510230:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002612b43f 214: 1< 0 - - > 2< 876544 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510233:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000f686f92b 00000080:00200000:2.0:1787827241.510234:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 215. 00000080:00200000:2.0:1787827241.510236:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 215 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510251:0:48388:0:(rw26.c:875:ll_write_end()) pos 880640, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510256:0:48388:0:(rw26.c:933:ll_write_end()) page@000000009b1bc559[3 00000000f7922827 2 1 000000000104970a 0 0 215] 00000080:00200000:2.0:1787827241.510260:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b218fc66 17ffffc0001001 3:0 ffff9a7886954910 215 lru 00000080:00200000:2.0:1787827241.510266:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006ba69ad5 215: 1< 0 - - > 2< 880640 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510268:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000009b1bc559 00000080:00200000:2.0:1787827241.510270:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 216. 00000080:00200000:2.0:1787827241.510272:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 216 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510285:0:48388:0:(rw26.c:875:ll_write_end()) pos 884736, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510289:0:48388:0:(rw26.c:933:ll_write_end()) page@000000008f44c4e3[3 00000000f7922827 2 1 000000000104970a 0 0 216] 00000080:00200000:2.0:1787827241.510293:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000ab8a5c6 17ffffc0001001 3:0 ffff9a7886954d98 216 lru 00000080:00200000:2.0:1787827241.510297:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000089b70ac4 216: 1< 0 - - > 2< 884736 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510298:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000008f44c4e3 00000080:00200000:2.0:1787827241.510299:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 217. 00000080:00200000:2.0:1787827241.510301:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 217 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510310:0:48388:0:(rw26.c:875:ll_write_end()) pos 888832, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510313:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a8510939[3 00000000f7922827 2 1 000000000104970a 0 0 217] 00000080:00200000:2.0:1787827241.510317:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000713e6179 17ffffc0001001 3:0 ffff9a78869543a0 217 lru 00000080:00200000:2.0:1787827241.510321:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000cf3a4df2 217: 1< 0 - - > 2< 888832 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510323:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a8510939 00000080:00200000:2.0:1787827241.510324:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 218. 00000080:00200000:2.0:1787827241.510325:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 218 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510335:0:48388:0:(rw26.c:875:ll_write_end()) pos 892928, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510340:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000060eb600c[3 00000000f7922827 2 1 000000000104970a 0 0 218] 00000080:00200000:2.0:1787827241.510343:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000083f4258f 17ffffc0001001 3:0 ffff9a78869540e8 218 lru 00000080:00200000:2.0:1787827241.510349:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001e0690ac 218: 1< 0 - - > 2< 892928 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510352:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000060eb600c 00000080:00200000:2.0:1787827241.510353:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 219. 00000080:00200000:2.0:1787827241.510356:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 219 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510382:0:48388:0:(rw26.c:875:ll_write_end()) pos 897024, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510387:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000009e5970c[3 00000000f7922827 2 1 000000000104970a 0 0 219] 00000080:00200000:2.0:1787827241.510389:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000799077c2 17ffffc0001021 2:0 ffff9a78a6760cb0 219 lru 00000080:00200000:2.0:1787827241.510393:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000cb2ff985 219: 1< 0 - - > 2< 897024 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510395:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000009e5970c 00000080:00200000:2.0:1787827241.510396:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 220. 00000080:00200000:2.0:1787827241.510397:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 220 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510408:0:48388:0:(rw26.c:875:ll_write_end()) pos 901120, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510411:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000037c8e104[3 00000000f7922827 2 1 000000000104970a 0 0 220] 00000080:00200000:2.0:1787827241.510413:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e2bc5c34 17ffffc0001001 3:0 ffff9a78a6760ae0 220 lru 00000080:00200000:2.0:1787827241.510416:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000009c0d5437 220: 1< 0 - - > 2< 901120 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510419:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000037c8e104 00000080:00200000:2.0:1787827241.510420:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 221. 00000080:00200000:2.0:1787827241.510421:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 221 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510429:0:48388:0:(rw26.c:875:ll_write_end()) pos 905216, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510432:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000aebcee72[3 00000000f7922827 2 1 000000000104970a 0 0 221] 00000080:00200000:2.0:1787827241.510434:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000a7920bce 17ffffc0001001 3:0 ffff9a78a6760658 221 lru 00000080:00200000:2.0:1787827241.510438:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000383ed15f 221: 1< 0 - - > 2< 905216 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510439:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000aebcee72 00000080:00200000:2.0:1787827241.510440:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 222. 00000080:00200000:2.0:1787827241.510441:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 222 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510449:0:48388:0:(rw26.c:875:ll_write_end()) pos 909312, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510452:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000bdda73ff[3 00000000f7922827 2 1 000000000104970a 0 0 222] 00000080:00200000:2.0:1787827241.510453:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006711e868 17ffffc0001001 3:0 ffff9a78a67609f8 222 lru 00000080:00200000:2.0:1787827241.510457:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000a961e927 222: 1< 0 - - > 2< 909312 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510458:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000bdda73ff 00000080:00200000:2.0:1787827241.510459:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 223. 00000080:00200000:2.0:1787827241.510460:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 223 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510469:0:48388:0:(rw26.c:875:ll_write_end()) pos 913408, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510472:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006ad91126[3 00000000f7922827 2 1 000000000104970a 0 0 223] 00000080:00200000:2.0:1787827241.510474:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000034bb7300 17ffffc0001001 3:0 ffff9a78a6760910 223 lru 00000080:00200000:2.0:1787827241.510478:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000814b873b 223: 1< 0 - - > 2< 913408 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510479:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006ad91126 00000080:00200000:2.0:1787827241.510480:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 224. 00000080:00200000:2.0:1787827241.510481:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 224 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510489:0:48388:0:(rw26.c:875:ll_write_end()) pos 917504, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510491:0:48388:0:(rw26.c:933:ll_write_end()) page@000000004feffe86[3 00000000f7922827 2 1 000000000104970a 0 0 224] 00000080:00200000:2.0:1787827241.510493:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000082953261 17ffffc0001001 3:0 ffff9a78a6760d98 224 lru 00000080:00200000:2.0:1787827241.510497:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000077c4cb6 224: 1< 0 - - > 2< 917504 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510498:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000004feffe86 00000080:00200000:2.0:1787827241.510499:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 225. 00000080:00200000:2.0:1787827241.510500:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 225 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510524:0:48388:0:(rw26.c:875:ll_write_end()) pos 921600, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510527:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000073f4d506[3 00000000f7922827 2 1 000000000104970a 0 0 225] 00000080:00200000:2.0:1787827241.510529:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c74eaf21 17ffffc0001001 3:0 ffff9a78a67603a0 225 lru 00000080:00200000:2.0:1787827241.510533:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000afd4fbf7 225: 1< 0 - - > 2< 921600 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510535:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000073f4d506 00000080:00200000:2.0:1787827241.510536:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 226. 00000080:00200000:2.0:1787827241.510537:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 226 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510546:0:48388:0:(rw26.c:875:ll_write_end()) pos 925696, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510549:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000090ab9f70[3 00000000f7922827 2 1 000000000104970a 0 0 226] 00000080:00200000:2.0:1787827241.510551:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000188bd3d8 17ffffc0001001 3:0 ffff9a78a67600e8 226 lru 00000080:00200000:2.0:1787827241.510555:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e06bfdc4 226: 1< 0 - - > 2< 925696 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510556:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000090ab9f70 00000080:00200000:2.0:1787827241.510557:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 227. 00000080:00200000:2.0:1787827241.510558:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 227 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510567:0:48388:0:(rw26.c:875:ll_write_end()) pos 929792, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510570:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000957c8ac6[3 00000000f7922827 2 1 000000000104970a 0 0 227] 00000080:00200000:2.0:1787827241.510571:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006198a27b 17ffffc0001001 3:0 ffff9a78a67602b8 227 lru 00000080:00200000:2.0:1787827241.510575:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d76cec63 227: 1< 0 - - > 2< 929792 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510576:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000957c8ac6 00000080:00200000:2.0:1787827241.510577:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 228. 00000080:00200000:2.0:1787827241.510578:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 228 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510587:0:48388:0:(rw26.c:875:ll_write_end()) pos 933888, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510589:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006c1e0275[3 00000000f7922827 2 1 000000000104970a 0 0 228] 00000080:00200000:2.0:1787827241.510591:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000097854124 17ffffc0001001 3:0 ffff9a78a67601d0 228 lru 00000080:00200000:2.0:1787827241.510595:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000002d9ec877 228: 1< 0 - - > 2< 933888 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510596:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006c1e0275 00000080:00200000:2.0:1787827241.510596:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 229. 00000080:00200000:2.0:1787827241.510597:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 229 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510606:0:48388:0:(rw26.c:875:ll_write_end()) pos 937984, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510609:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000008630992[3 00000000f7922827 2 1 000000000104970a 0 0 229] 00000080:00200000:2.0:1787827241.510610:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b11f2863 17ffffc0001001 3:0 ffff9a78a6760740 229 lru 00000080:00200000:2.0:1787827241.510614:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000041b990a6 229: 1< 0 - - > 2< 937984 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510616:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000008630992 00000080:00200000:2.0:1787827241.510616:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 230. 00000080:00200000:2.0:1787827241.510617:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 230 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510626:0:48388:0:(rw26.c:875:ll_write_end()) pos 942080, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510629:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000dcb68fe[3 00000000f7922827 2 1 000000000104970a 0 0 230] 00000080:00200000:2.0:1787827241.510630:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007d5c5404 17ffffc0001001 3:0 ffff9a78a6760e80 230 lru 00000080:00200000:2.0:1787827241.510634:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000093569346 230: 1< 0 - - > 2< 942080 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510635:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000dcb68fe 00000080:00200000:2.0:1787827241.510635:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 231. 00000080:00200000:2.0:1787827241.510636:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 231 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510645:0:48388:0:(rw26.c:875:ll_write_end()) pos 946176, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510647:0:48388:0:(rw26.c:933:ll_write_end()) page@000000006fa57646[3 00000000f7922827 2 1 000000000104970a 0 0 231] 00000080:00200000:2.0:1787827241.510649:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009793dd97 17ffffc0001001 3:0 ffff9a78a6760488 231 lru 00000080:00200000:2.0:1787827241.510653:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000007d128a3c 231: 1< 0 - - > 2< 946176 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510655:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000006fa57646 00000080:00200000:2.0:1787827241.510656:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 232. 00000080:00200000:2.0:1787827241.510657:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 232 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510665:0:48388:0:(rw26.c:875:ll_write_end()) pos 950272, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510668:0:48388:0:(rw26.c:933:ll_write_end()) page@000000002d2c8c1d[3 00000000f7922827 2 1 000000000104970a 0 0 232] 00000080:00200000:2.0:1787827241.510669:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000045ca9edc 17ffffc0001001 3:0 ffff9a78a6760bc8 232 lru 00000080:00200000:2.0:1787827241.510673:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001a41a674 232: 1< 0 - - > 2< 950272 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510674:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000002d2c8c1d 00000080:00200000:2.0:1787827241.510675:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 233. 00000080:00200000:2.0:1787827241.510676:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 233 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510685:0:48388:0:(rw26.c:875:ll_write_end()) pos 954368, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510687:0:48388:0:(rw26.c:933:ll_write_end()) page@000000003dec2a4a[3 00000000f7922827 2 1 000000000104970a 0 0 233] 00000080:00200000:2.0:1787827241.510689:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000007095b960 17ffffc0001001 3:0 ffff9a78a6760570 233 lru 00000080:00200000:2.0:1787827241.510692:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000dd9338fe 233: 1< 0 - - > 2< 954368 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510693:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000003dec2a4a 00000080:00200000:2.0:1787827241.510694:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 234. 00000080:00200000:2.0:1787827241.510695:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 234 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510708:0:48388:0:(rw26.c:875:ll_write_end()) pos 958464, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510711:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000068b047b7[3 00000000f7922827 2 1 000000000104970a 0 0 234] 00000080:00200000:2.0:1787827241.510713:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006046de54 17ffffc0001021 2:0 ffff9a78a6760828 234 lru 00000080:00200000:2.0:1787827241.510717:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000070e01284 234: 1< 0 - - > 2< 958464 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510719:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000068b047b7 00000080:00200000:2.0:1787827241.510719:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 235. 00000080:00200000:2.0:1787827241.510720:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 235 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510729:0:48388:0:(rw26.c:875:ll_write_end()) pos 962560, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510732:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000c8ca4584[3 00000000f7922827 2 1 000000000104970a 0 0 235] 00000080:00200000:2.0:1787827241.510735:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000036cc0436 17ffffc0001001 3:0 ffff9a78a6760000 235 lru 00000080:00200000:2.0:1787827241.510741:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c407c607 235: 1< 0 - - > 2< 962560 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510743:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000c8ca4584 00000080:00200000:2.0:1787827241.510745:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 236. 00000080:00200000:2.0:1787827241.510747:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 236 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510770:0:48388:0:(rw26.c:875:ll_write_end()) pos 966656, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510775:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000865efa86[3 00000000f7922827 2 1 000000000104970a 0 0 236] 00000080:00200000:2.0:1787827241.510778:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000fac22a3c 17ffffc0001001 3:0 ffff9a7890a33740 236 lru 00000080:00200000:2.0:1787827241.510785:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000febfa4fe 236: 1< 0 - - > 2< 966656 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510787:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000865efa86 00000080:00200000:2.0:1787827241.510789:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 237. 00000080:00200000:2.0:1787827241.510791:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 237 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510808:0:48388:0:(rw26.c:875:ll_write_end()) pos 970752, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510813:0:48388:0:(rw26.c:933:ll_write_end()) page@000000008115db88[3 00000000f7922827 2 1 000000000104970a 0 0 237] 00000080:00200000:2.0:1787827241.510816:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e2f247da 17ffffc0001001 3:0 ffff9a7890a33e80 237 lru 00000080:00200000:2.0:1787827241.510822:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000006066d02f 237: 1< 0 - - > 2< 970752 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510825:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000008115db88 00000080:00200000:2.0:1787827241.510826:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 238. 00000080:00200000:2.0:1787827241.510848:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 238 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510866:0:48388:0:(rw26.c:875:ll_write_end()) pos 974848, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510872:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005079a162[3 00000000f7922827 2 1 000000000104970a 0 0 238] 00000080:00200000:2.0:1787827241.510875:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000005e791c44 17ffffc0001001 3:0 ffff9a7890a33488 238 lru 00000080:00200000:2.0:1787827241.510881:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000f659b6cd 238: 1< 0 - - > 2< 974848 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510884:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005079a162 00000080:00200000:2.0:1787827241.510885:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 239. 00000080:00200000:2.0:1787827241.510888:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 239 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510906:0:48388:0:(rw26.c:875:ll_write_end()) pos 978944, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510911:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000fce23d40[3 00000000f7922827 2 1 000000000104970a 0 0 239] 00000080:00200000:2.0:1787827241.510914:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c0a09db3 17ffffc0001001 3:0 ffff9a7890a33bc8 239 lru 00000080:00200000:2.0:1787827241.510920:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000fa5c6bca 239: 1< 0 - - > 2< 978944 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510922:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000fce23d40 00000080:00200000:2.0:1787827241.510923:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 240. 00000080:00200000:2.0:1787827241.510925:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 240 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510944:0:48388:0:(rw26.c:875:ll_write_end()) pos 983040, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510949:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e2dd7c97[3 00000000f7922827 2 1 000000000104970a 0 0 240] 00000080:00200000:2.0:1787827241.510952:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000020517d03 17ffffc0001001 3:0 ffff9a7890a33570 240 lru 00000080:00200000:2.0:1787827241.510958:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000047f1445c 240: 1< 0 - - > 2< 983040 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.510960:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e2dd7c97 00000080:00200000:2.0:1787827241.510962:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 241. 00000080:00200000:2.0:1787827241.510964:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 241 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.510983:0:48388:0:(rw26.c:875:ll_write_end()) pos 987136, len 4096, copied 4096 00000080:00200000:2.0:1787827241.510988:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000845bb9cd[3 00000000f7922827 2 1 000000000104970a 0 0 241] 00000080:00200000:2.0:1787827241.510991:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000939ee468 17ffffc0001001 3:0 ffff9a7890a33828 241 lru 00000080:00200000:2.0:1787827241.510997:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000015d0e0cf 241: 1< 0 - - > 2< 987136 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511000:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000845bb9cd 00000080:00200000:2.0:1787827241.511001:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 242. 00000080:00200000:2.0:1787827241.511004:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 242 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511022:0:48388:0:(rw26.c:875:ll_write_end()) pos 991232, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511027:0:48388:0:(rw26.c:933:ll_write_end()) page@000000002174b32a[3 00000000f7922827 2 1 000000000104970a 0 0 242] 00000080:00200000:2.0:1787827241.511030:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000094b2f2a6 17ffffc0001001 3:0 ffff9a7890a33000 242 lru 00000080:00200000:2.0:1787827241.511036:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000cf0127b2 242: 1< 0 - - > 2< 991232 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511039:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000002174b32a 00000080:00200000:2.0:1787827241.511040:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 243. 00000080:00200000:2.0:1787827241.511042:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 243 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511060:0:48388:0:(rw26.c:875:ll_write_end()) pos 995328, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511066:0:48388:0:(rw26.c:933:ll_write_end()) page@0000000091db1ee9[3 00000000f7922827 2 1 000000000104970a 0 0 243] 00000080:00200000:2.0:1787827241.511069:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b61def5d 17ffffc0001001 3:0 ffff9a7890a33cb0 243 lru 00000080:00200000:2.0:1787827241.511075:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000010e7d92e 243: 1< 0 - - > 2< 995328 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511078:0:48388:0:(rw26.c:933:ll_write_end()) end page@0000000091db1ee9 00000080:00200000:2.0:1787827241.511079:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 244. 00000080:00200000:2.0:1787827241.511081:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 244 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511098:0:48388:0:(rw26.c:875:ll_write_end()) pos 999424, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511103:0:48388:0:(rw26.c:933:ll_write_end()) page@000000005e3b1f73[3 00000000f7922827 2 1 000000000104970a 0 0 244] 00000080:00200000:2.0:1787827241.511107:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000a07f4b30 17ffffc0001001 3:0 ffff9a7890a33ae0 244 lru 00000080:00200000:2.0:1787827241.511113:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000992089a6 244: 1< 0 - - > 2< 999424 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511116:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000005e3b1f73 00000080:00200000:2.0:1787827241.511117:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 245. 00000080:00200000:2.0:1787827241.511120:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 245 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511135:0:48388:0:(rw26.c:875:ll_write_end()) pos 1003520, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511140:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000cc27a9f1[3 00000000f7922827 2 1 000000000104970a 0 0 245] 00000080:00200000:2.0:1787827241.511143:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000335c85e1 17ffffc0001001 3:0 ffff9a7890a33658 245 lru 00000080:00200000:2.0:1787827241.511150:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e8821e31 245: 1< 0 - - > 2< 1003520 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511152:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000cc27a9f1 00000080:00200000:2.0:1787827241.511154:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 246. 00000080:00200000:2.0:1787827241.511156:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 246 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511172:0:48388:0:(rw26.c:875:ll_write_end()) pos 1007616, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511177:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000374e294c[3 00000000f7922827 2 1 000000000104970a 0 0 246] 00000080:00200000:2.0:1787827241.511180:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000e1a1f19b 17ffffc0001001 3:0 ffff9a7890a339f8 246 lru 00000080:00200000:2.0:1787827241.511186:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d278118a 246: 1< 0 - - > 2< 1007616 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511189:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000374e294c 00000080:00200000:2.0:1787827241.511190:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 247. 00000080:00200000:2.0:1787827241.511192:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 247 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511209:0:48388:0:(rw26.c:875:ll_write_end()) pos 1011712, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511214:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000e7c436f4[3 00000000f7922827 2 1 000000000104970a 0 0 247] 00000080:00200000:2.0:1787827241.511217:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000006f6c4e56 17ffffc0001001 3:0 ffff9a7890a33910 247 lru 00000080:00200000:2.0:1787827241.511223:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000ffa89f40 247: 1< 0 - - > 2< 1011712 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511226:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000e7c436f4 00000080:00200000:2.0:1787827241.511228:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 248. 00000080:00200000:2.0:1787827241.511230:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 248 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511250:0:48388:0:(rw26.c:875:ll_write_end()) pos 1015808, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511255:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000343e7d6f[3 00000000f7922827 2 1 000000000104970a 0 0 248] 00000080:00200000:2.0:1787827241.511258:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000b2d363e1 17ffffc0001001 3:0 ffff9a7890a33d98 248 lru 00000080:00200000:2.0:1787827241.511265:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000d319e345 248: 1< 0 - - > 2< 1015808 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511267:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000343e7d6f 00000080:00200000:2.0:1787827241.511268:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 249. 00000080:00200000:2.0:1787827241.511271:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 249 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511292:0:48388:0:(rw26.c:875:ll_write_end()) pos 1019904, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511298:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000ca72add5[3 00000000f7922827 2 1 000000000104970a 0 0 249] 00000080:00200000:2.0:1787827241.511301:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000000e5b482d 17ffffc0001021 2:0 ffff9a7890a333a0 249 lru 00000080:00200000:2.0:1787827241.511307:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000c998ae1b 249: 1< 0 - - > 2< 1019904 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511310:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000ca72add5 00000080:00200000:2.0:1787827241.511311:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 250. 00000080:00200000:2.0:1787827241.511314:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 250 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511332:0:48388:0:(rw26.c:875:ll_write_end()) pos 1024000, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511336:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000dcd500ad[3 00000000f7922827 2 1 000000000104970a 0 0 250] 00000080:00200000:2.0:1787827241.511340:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000383a56d5 17ffffc0001001 3:0 ffff9a7890a330e8 250 lru 00000080:00200000:2.0:1787827241.511346:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000014febad9 250: 1< 0 - - > 2< 1024000 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511348:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000dcd500ad 00000080:00200000:2.0:1787827241.511350:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 251. 00000080:00200000:2.0:1787827241.511352:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 251 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511370:0:48388:0:(rw26.c:875:ll_write_end()) pos 1028096, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511376:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000350f10d0[3 00000000f7922827 2 1 000000000104970a 0 0 251] 00000080:00200000:2.0:1787827241.511379:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d7aa7da8 17ffffc0001001 3:0 ffff9a7890a332b8 251 lru 00000080:00200000:2.0:1787827241.511385:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000294e9e01 251: 1< 0 - - > 2< 1028096 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511388:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000350f10d0 00000080:00200000:2.0:1787827241.511389:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 252. 00000080:00200000:2.0:1787827241.511392:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 252 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511409:0:48388:0:(rw26.c:875:ll_write_end()) pos 1032192, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511415:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000a4f0b9b2[3 00000000f7922827 2 1 000000000104970a 0 0 252] 00000080:00200000:2.0:1787827241.511418:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000c79042ab 17ffffc0001001 3:0 ffff9a7890a331d0 252 lru 00000080:00200000:2.0:1787827241.511424:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000054924d33 252: 1< 0 - - > 2< 1032192 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511427:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000a4f0b9b2 00000080:00200000:2.0:1787827241.511428:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 253. 00000080:00200000:2.0:1787827241.511431:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 253 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511458:0:48388:0:(rw26.c:875:ll_write_end()) pos 1036288, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511464:0:48388:0:(rw26.c:933:ll_write_end()) page@00000000383a57e6[3 00000000f7922827 2 1 000000000104970a 0 0 253] 00000080:00200000:2.0:1787827241.511468:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @000000009cdbd43a 17ffffc0001001 3:0 ffff9a7889128658 253 lru 00000080:00200000:2.0:1787827241.511474:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@000000001a26bfc4 253: 1< 0 - - > 2< 1036288 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511477:0:48388:0:(rw26.c:933:ll_write_end()) end page@00000000383a57e6 00000080:00200000:2.0:1787827241.511478:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 254. 00000080:00200000:2.0:1787827241.511481:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 254 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511499:0:48388:0:(rw26.c:875:ll_write_end()) pos 1040384, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511518:0:48388:0:(rw26.c:933:ll_write_end()) page@000000000e969624[3 00000000f7922827 2 1 000000000104970a 0 0 254] 00000080:00200000:2.0:1787827241.511522:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @00000000d0de437e 17ffffc0001001 3:0 ffff9a78891289f8 254 lru 00000080:00200000:2.0:1787827241.511528:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@00000000e262c33c 254: 1< 0 - - > 2< 1040384 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511531:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000000e969624 00000080:00200000:2.0:1787827241.511532:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 255. 00000080:00200000:2.0:1787827241.511535:0:48388:0:(rw26.c:681:ll_write_begin()) Writing 255 of 0 to 4096 bytes 00000080:00200000:2.0:1787827241.511556:0:48388:0:(rw26.c:875:ll_write_end()) pos 1044480, len 4096, copied 4096 00000080:00200000:2.0:1787827241.511562:0:48388:0:(rw26.c:933:ll_write_end()) page@000000007ccb25a1[3 00000000f7922827 2 1 000000000104970a 0 0 255] 00000080:00200000:2.0:1787827241.511565:0:48388:0:(rw26.c:933:ll_write_end()) vmpage @0000000094e04292 17ffffc0001001 3:0 ffff9a7889128910 255 lru 00000080:00200000:2.0:1787827241.511571:0:48388:0:(rw26.c:933:ll_write_end()) osc-page@0000000010447e7f 255: 1< 0 - - > 2< 1044480 0 0 0x0 0x0 | 0000000043fddcb7 0000000070324f39 > 3< 0 0 > 4< 0 0 8 1552384 - | - - + - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1787827241.511574:0:48388:0:(rw26.c:933:ll_write_end()) end page@000000007ccb25a1 00000080:00200000:2.0:1787827241.511575:0:48388:0:(rw26.c:933:ll_write_end()) queued cl_page: 256. 00000080:00200000:2.0:1787827241.511580:0:48388:0:(vvp_io.c:1214:vvp_io_write_commit()) commit async pages: 256, from 0, to 4096 prio 0 00000008:00010000:2.0:1787827241.511638:0:48388:0:(osc_cache.c:858:osc_extent_find()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511675:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511688:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511699:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511709:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511721:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511731:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511742:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511752:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511763:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511774:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511785:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511796:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511805:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511816:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.511855:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.511866:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511880:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511894:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511910:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511921:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511932:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511942:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511953:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511964:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511974:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511984:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.511994:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512005:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512016:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512025:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.512041:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.512048:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512058:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512068:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512079:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512089:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512106:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512116:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512126:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512136:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512146:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512156:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512165:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512175:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512185:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512196:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.512211:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.512218:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512228:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512239:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512249:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512259:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512269:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512279:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512296:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512308:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512318:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512328:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512338:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512348:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512358:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512368:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.512382:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.512390:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512401:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512411:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512421:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512431:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512442:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512452:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512462:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512472:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512489:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512500:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512527:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512538:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512549:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512559:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.512574:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.512581:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512591:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512608:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512618:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512628:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512639:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512649:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512659:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512669:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512679:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512689:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512699:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512711:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512721:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512732:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.512746:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.512754:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512764:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512774:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512784:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512801:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512812:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512822:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512851:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512862:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512872:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512882:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512893:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512903:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512913:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512924:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.512939:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.512946:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512957:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512967:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512977:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512987:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.512997:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513014:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513024:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513034:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513045:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513056:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513066:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513077:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513087:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513096:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.513111:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.513117:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513128:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513138:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513148:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513158:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513168:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513178:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513188:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513206:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513216:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513227:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513237:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513247:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513257:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513268:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.513283:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.513290:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513306:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513317:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513328:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513338:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513347:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513357:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513367:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513377:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513386:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513397:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513405:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513415:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513426:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513435:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.513450:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.513456:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513466:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513476:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513493:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513521:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513532:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513542:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513553:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513563:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513573:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513584:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513594:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513605:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513615:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513626:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.513641:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.513648:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513658:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513668:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513678:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513688:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513704:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513715:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513726:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513736:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513746:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513757:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513767:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513778:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513788:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513800:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.513814:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.513821:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513852:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513863:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513874:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513885:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513896:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513907:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513925:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513934:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513945:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513955:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513965:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513975:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513985:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.513996:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.514012:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.514019:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514030:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514041:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514052:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514063:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514073:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514084:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514094:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514104:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514121:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514132:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514142:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514153:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514163:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514174:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.514188:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.514195:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514205:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514221:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514231:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514241:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514252:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514262:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514272:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514282:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514291:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514300:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514311:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514321:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514332:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514343:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.514358:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.514364:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514374:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514384:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514393:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514409:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514419:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514429:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514439:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514449:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514459:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514469:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514480:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514490:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514516:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514526:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.514540:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.514545:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514554:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514564:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514574:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514583:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514592:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514605:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514613:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514621:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514629:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514637:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514646:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514653:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514661:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1787827241.514669:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.514683:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 15, dirtied 15 00000008:00010000:2.0:1787827241.514688:0:48388:0:(osc_cache.c:2567:osc_queue_async_io()) ### extent: 000000001a2faca1 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1787827241.514696:0:48388:0:(vvp_io.c:1127:vvp_set_batch_dirty()) mapping ffff9a789ded75f8, count 1, dirtied 1 00000080:00200000:2.0:1787827241.514701:0:48388:0:(vvp_io.c:1237:vvp_io_write_commit()) Committed 256 pages 1048576 bytes, tot: 1048576 00000080:00200000:2.0:1787827241.514705:0:48388:0:(vvp_io.c:1421:vvp_io_write_start()) f2: write bytes 0, result: 1048576 00000020:00010000:2.0:1787827241.514726:0:48388:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9a78879513f8 (ffff9a78899dbe18/0) at cl_lock_release():245 00000020:00010000:2.0:1787827241.514730:0:48388:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a78879513f8 (ffff9a78899dbe18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1787827241.514734:0:48388:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9a7890c3c8b0 (ffff9a78899db878/1) at cl_lock_cancel():142 00010000:00010000:2.0:1787827241.514743:0:48388:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 5/0,1 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827241.514756:0:48388:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f85cb279/0xebdcd48a4298ef63 lrc: 4/0,0 mode: PW/PW res: [0x2c0000400:0xa9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x4059aa0aef8dcde9 expref: -99 pid: 48388 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:2.0:1787827241.514776:0:48388:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a78879513f8 (ffff9a78899dbe18/0) at cl_lock_fini():72 00000020:00010000:2.0:1787827241.514780:0:48388:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9a7890c3c8b0 (ffff9a78899dbe18/1) at cl_lock_fini():72 00000080:00200000:2.0:1787827241.514792:0:48388:0:(vvp_io.c:533:vvp_io_advance()) advancing 1048576 bytes 00000080:00200000:2.0:1787827241.514802:0:48388:0:(file.c:2207:ll_file_io_generic()) Range unlock [0, 255] 00000080:00200000:2.0:1787827241.514813:0:48388:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x51:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.514818:0:48388:0:(file.c:2240:ll_file_io_generic()) f2: 2 io complete with rc: 0, result: 1048576, restart: 0 00000080:00200000:2.0:1787827241.514850:0:48388:0:(file.c:1775:ll_merge_attr_nolock()) [0x240001b71:0x51:0x0] updating i_size 1048576 i_blocks 0 00000080:00200000:2.0:1787827241.514858:0:48388:0:(file.c:2326:ll_file_io_generic()) iot: 2, result: 1048576 00000080:00200008:2.0:1787827241.514900:0:48388:0:(file.c:479:ll_file_release()) START file f2:[0x240001b71:0x51:0x0](ffff9a789ded7448), flags 100001 00000002:00100000:2.0:1787827241.514990:0:48388:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9a7886bae940 x1874671476649216/t17179869533(17179869533) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787827241.515011:0:48388:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787827241.515018:0:48388:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886baf0c0 x1874671476649600/t0(0) o35->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787827241.515048:0:48388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886baf0c0 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48388:1874671476649600:192.168.203.140@tcp:35:dd.0 00000100:00100000:2.0:1787827241.515153:0:48388:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.516564:0:48388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886baf0c0 pname:cluuid:pid:xid:nid:opc:job dd:97196825-2cb9-49ba-a47d-1455a997ce3a:48388:1874671476649600:192.168.203.140@tcp:35:dd.0 00000002:00100000:2.0:1787827241.516589:0:48388:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9a7886bae940 x1874671476649216/t17179869533(17179869533) o101->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 576/664 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787827241.522260:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.522287:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.522306:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.522323:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.522334:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.522343:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.522348:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.522357:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.522363:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.522371:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.522391:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.522399:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.522404:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.522406:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.522409:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.522418:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub0, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.522421:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.522425:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.522429:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub0, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.522432:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub0, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.522491:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.522515:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bada40 x1874671476649856/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.522539:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bada40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476649856:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.522635:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.524319:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bada40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476649856:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.524351:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x52:0x0](ffff9a789ded5348) 00000080:00200000:2.0:1787827241.524385:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub0, unhashed 1 00000080:00200000:2.0:1787827241.524411:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.524418:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.524426:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.524435:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.524442:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.524445:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.524447:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.524450:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.524453:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.524459:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.524466:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.524469:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.524471:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.524473:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.524475:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.524482:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub1, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.524485:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.524488:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.524490:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub1, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.524492:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub1, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.524532:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.524540:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bade00 x1874671476650112/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.524556:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bade00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650112:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.524619:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.525699:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bade00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650112:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.525718:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x53:0x0](ffff9a789ded5b88) 00000080:00200000:2.0:1787827241.525732:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub1, unhashed 1 00000080:00200000:2.0:1787827241.525751:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.525759:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.525766:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.525774:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.525780:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.525783:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.525785:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.525789:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.525792:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.525797:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.525804:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.525807:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.525810:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.525812:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.525814:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.525821:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub2, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.525823:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.525827:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.525972:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub2, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.525975:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub2, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.526059:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.526065:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bacb40 x1874671476650368/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.526076:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bacb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650368:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.526113:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.527214:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bacb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650368:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.527234:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x54:0x0](ffff9a789ded2a08) 00000080:00200000:2.0:1787827241.527248:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub2, unhashed 1 00000080:00200000:2.0:1787827241.527266:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.527274:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.527281:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.527289:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.527295:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.527298:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.527300:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.527303:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.527306:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.527317:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.527328:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.527332:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.527337:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.527340:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.527344:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.527354:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub3, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.527358:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.527363:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.527367:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub3, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.527371:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub3, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.527396:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.527402:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bae580 x1874671476650624/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.527413:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bae580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650624:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.527451:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.528515:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bae580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650624:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.528531:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x55:0x0](ffff9a789ded1988) 00000080:00200000:2.0:1787827241.528543:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub3, unhashed 1 00000080:00200000:2.0:1787827241.528560:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.528567:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.528573:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.528581:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.528586:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.528589:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.528591:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.528594:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.528597:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.528602:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.528609:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.528611:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.528614:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.528615:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.528617:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.528624:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub4, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.528626:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.528629:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.528631:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub4, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.528633:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub4, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.528650:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.528656:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886baf840 x1874671476650880/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.528671:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886baf840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650880:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.528705:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.529658:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886baf840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476650880:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.529678:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x56:0x0](ffff9a789ded0908) 00000080:00200000:2.0:1787827241.529692:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub4, unhashed 1 00000080:00200000:2.0:1787827241.529712:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.529720:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.529727:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.529735:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.529741:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.529744:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.529746:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.529749:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.529752:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.529758:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.529769:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.529774:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.529778:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.529782:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.529785:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.529796:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub5, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.529800:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.529805:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.529809:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub5, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.529813:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub5, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.530190:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.530199:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bad2c0 x1874671476651136/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.530217:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bad2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651136:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.530267:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.531348:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bad2c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651136:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.531373:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x57:0x0](ffff9a789ded63c8) 00000080:00200000:2.0:1787827241.531389:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub5, unhashed 1 00000080:00200000:2.0:1787827241.531411:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.531419:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.531426:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.531435:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.531441:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.531444:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.531446:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.531451:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.531453:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.531459:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.531466:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.531469:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.531472:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.531474:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.531475:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.531482:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub6, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.531484:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.531487:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.531490:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub6, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.531492:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub6, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.531530:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.531536:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bac780 x1874671476651392/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.531548:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bac780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651392:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.531587:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.532539:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bac780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651392:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.532558:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x58:0x0](ffff9a789ded4b08) 00000080:00200000:2.0:1787827241.532571:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub6, unhashed 1 00000080:00200000:2.0:1787827241.532588:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.532596:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.532602:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.532609:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.532615:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.532618:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.532620:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.532623:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.532625:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.532631:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.532637:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.532640:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.532642:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.532644:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.532646:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.532652:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub7, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.532654:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.532658:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.532660:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub7, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.532662:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub7, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.532680:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.532685:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bae1c0 x1874671476651648/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.532695:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bae1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651648:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.532750:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.533671:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bae1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651648:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.533688:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x59:0x0](ffff9a789ded3a88) 00000080:00200000:2.0:1787827241.533703:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub7, unhashed 1 00000080:00200000:2.0:1787827241.533723:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.533731:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.533738:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.533746:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.533752:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.533755:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.533757:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.533760:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.533763:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.533769:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.533775:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.533778:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.533781:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.533783:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.533785:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.533795:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub8, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.533797:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.533801:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.533803:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub8, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.533805:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub8, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.533880:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.533887:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bafc00 x1874671476651904/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.533899:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bafc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651904:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.533936:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.534814:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bafc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476651904:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.534875:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x5a:0x0](ffff9a789dedf448) 00000080:00200000:2.0:1787827241.534899:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub8, unhashed 1 00000080:00200000:2.0:1787827241.534916:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.534923:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.534929:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.534936:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.534941:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.534944:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.534946:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.534950:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.534952:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.534958:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.534965:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.534967:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.534969:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.534971:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.534973:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.534979:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub9, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.534981:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.534984:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.534987:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub9, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.534989:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub9, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.535007:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.535012:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bac3c0 x1874671476652160/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.535022:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bac3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652160:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.535060:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.536309:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bac3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652160:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.536344:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x5b:0x0](ffff9a789dedd348) 00000080:00200000:2.0:1787827241.536369:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub9, unhashed 1 00000080:00200000:2.0:1787827241.536403:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.536417:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.536429:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.536441:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.536448:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.536452:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.536455:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.536461:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.536463:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.536472:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.536484:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.536489:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.536493:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.536496:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.536499:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.536525:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub10, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.536528:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.536533:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.536536:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub10, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.536540:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub10, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.536578:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.536586:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a7886bac000 x1874671476652416/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.536603:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886bac000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652416:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.536665:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.537739:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886bac000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652416:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.537756:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x5c:0x0](ffff9a789deddb88) 00000080:00200000:2.0:1787827241.537769:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub10, unhashed 1 00000080:00200000:2.0:1787827241.537787:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.537793:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.537800:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.537806:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.537811:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.537814:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.537816:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.537819:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.537822:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.537860:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.537868:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.537871:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.537874:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.537876:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.537878:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.537885:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub11, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.537887:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.537890:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.537893:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub11, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.537895:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub11, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.537914:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.537919:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d0780 x1874671476652672/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.537930:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d0780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652672:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.537964:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.538910:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d0780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652672:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.538926:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x5d:0x0](ffff9a789dedaa08) 00000080:00200000:2.0:1787827241.538938:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub11, unhashed 1 00000080:00200000:2.0:1787827241.538953:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.538959:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.538966:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.538973:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.538977:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.538980:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.538982:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.538986:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.538988:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.538994:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.539000:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.539003:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.539005:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.539006:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.539008:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.539014:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub12, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.539016:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.539019:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.539022:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub12, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.539024:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub12, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.539041:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.539045:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d3480 x1874671476652928/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.539055:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d3480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652928:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.539088:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.540038:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d3480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476652928:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.540052:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x5e:0x0](ffff9a789ded9988) 00000080:00200000:2.0:1787827241.540065:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub12, unhashed 1 00000080:00200000:2.0:1787827241.540080:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.540087:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.540093:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.540099:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.540105:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.540108:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.540109:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.540112:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.540115:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.540120:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.540126:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.540128:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.540131:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.540132:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.540134:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.540140:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub13, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.540142:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.540145:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.540148:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub13, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.540150:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub13, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.540167:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.540171:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d3c00 x1874671476653184/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.540186:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d3c00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653184:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.540236:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.541210:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d3c00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653184:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.541228:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x5f:0x0](ffff9a789ded8908) 00000080:00200000:2.0:1787827241.541242:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub13, unhashed 1 00000080:00200000:2.0:1787827241.541260:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.541268:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.541274:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.541283:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.541289:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.541292:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.541294:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.541297:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.541299:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.541306:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.541312:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.541315:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.541317:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.541320:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.541322:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.541329:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub14, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.541332:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.541335:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.541337:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub14, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.541339:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub14, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.541358:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.541364:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d2d00 x1874671476653440/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.541374:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d2d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653440:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.541413:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.542426:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d2d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653440:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.542449:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x60:0x0](ffff9a789dede3c8) 00000080:00200000:2.0:1787827241.542486:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub14, unhashed 1 00000080:00200000:2.0:1787827241.542526:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.542537:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.542547:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.542559:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.542565:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.542569:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.542571:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.542575:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.542578:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.542584:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.542591:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.542594:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.542597:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.542598:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.542600:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.542607:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub15, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.542610:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.542613:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.542615:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub15, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.542617:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub15, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.542637:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.542643:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d0f00 x1874671476653696/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.542653:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d0f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653696:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.542700:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.543616:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d0f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653696:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.543634:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x61:0x0](ffff9a789dedcb08) 00000080:00200000:2.0:1787827241.543647:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub15, unhashed 1 00000080:00200000:2.0:1787827241.543664:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.543672:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.543679:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.543687:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.543693:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.543696:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.543698:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.543701:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.543704:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.543710:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.543716:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.543720:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.543725:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.543728:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.543731:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.543740:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub16, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.543744:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.543749:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.543752:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub16, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.543756:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub16, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.543792:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.543799:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d3840 x1874671476653952/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.543809:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d3840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653952:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.543860:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.544846:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d3840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476653952:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.544867:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x62:0x0](ffff9a789dedba88) 00000080:00200000:2.0:1787827241.544882:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub16, unhashed 1 00000080:00200000:2.0:1787827241.544902:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.544909:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.544916:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.544925:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.544930:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.544934:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.544936:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.544939:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.544942:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.544948:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.544954:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.544957:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.544960:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.544962:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.544964:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.544970:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub17, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.544972:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.544976:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.544978:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub17, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.544980:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub17, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.544998:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.545003:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d2580 x1874671476654208/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.545014:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d2580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654208:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.545050:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.546107:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d2580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654208:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.546121:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x63:0x0](ffff9a789ded9148) 00000080:00200000:2.0:1787827241.546133:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub17, unhashed 1 00000080:00200000:2.0:1787827241.546148:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.546154:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.546160:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.546168:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.546172:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.546175:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.546177:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.546181:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.546184:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.546189:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.546196:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.546198:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.546200:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.546202:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.546204:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.546209:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub18, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.546212:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.546215:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.546217:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub18, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.546219:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub18, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.546236:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.546241:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d30c0 x1874671476654464/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.546250:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d30c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654464:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.546283:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.547261:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d30c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654464:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.547279:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x64:0x0](ffff9a789ded80c8) 00000080:00200000:2.0:1787827241.547292:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub18, unhashed 1 00000080:00200000:2.0:1787827241.547310:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.547318:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.547325:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.547333:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.547339:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.547342:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.547344:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.547347:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.547350:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.547356:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.547362:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.547365:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.547367:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.547369:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.547370:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.547377:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub19, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.547380:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.547383:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.547385:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub19, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.547387:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub19, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.547406:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.547411:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d1e00 x1874671476654720/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.547422:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d1e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654720:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.547458:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.548357:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d1e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654720:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.548371:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x65:0x0](ffff9a789deda1c8) 00000080:00200000:2.0:1787827241.548384:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub19, unhashed 1 00000080:00200000:2.0:1787827241.548399:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.548406:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.548412:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.548420:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.548428:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.548432:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.548435:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.548439:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.548443:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.548452:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.548462:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.548467:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.548472:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.548474:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.548476:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.548483:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub20, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.548485:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.548489:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.548491:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub20, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.548493:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub20, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.548526:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.548531:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d0b40 x1874671476654976/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.548542:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d0b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654976:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.548575:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.549589:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d0b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476654976:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.549608:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x66:0x0](ffff9a789dedb248) 00000080:00200000:2.0:1787827241.549621:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub20, unhashed 1 00000080:00200000:2.0:1787827241.549639:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.549646:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.549653:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.549661:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.549667:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.549670:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.549672:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.549676:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.549679:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.549685:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.549691:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.549694:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.549696:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.549698:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.549699:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.549706:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub21, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.549709:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.549712:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.549714:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub21, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.549716:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub21, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.549742:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.549749:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d03c0 x1874671476655232/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.549765:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d03c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476655232:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.549807:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.550773:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d03c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476655232:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.550794:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x67:0x0](ffff9a789dedec08) 00000080:00200000:2.0:1787827241.550811:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub21, unhashed 1 00000080:00200000:2.0:1787827241.550851:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.550861:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.550868:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.550877:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.550882:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.550886:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.550888:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.550891:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.550894:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.550900:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.550907:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.550910:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.550912:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.550914:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.550916:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.550924:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub22, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.550926:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.550929:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.550931:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub22, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.550934:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub22, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.550952:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.550958:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d1a40 x1874671476655488/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.550969:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d1a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476655488:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.551004:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.551960:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d1a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476655488:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.551977:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x68:0x0](ffff9a789dedc2c8) 00000080:00200000:2.0:1787827241.551989:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub22, unhashed 1 00000080:00200000:2.0:1787827241.552005:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.552012:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.552018:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.552033:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.552038:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.552041:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.552043:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.552046:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.552049:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.552054:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.552060:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.552063:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.552066:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.552067:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.552069:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.552076:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub23, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.552079:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.552082:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.552084:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub23, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.552086:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub23, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.552103:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.552108:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d0000 x1874671476655744/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.552118:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d0000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476655744:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.552169:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.553160:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d0000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476655744:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.553196:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x69:0x0](ffff9a789dff6c08) 00000080:00200000:2.0:1787827241.553214:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub23, unhashed 1 00000080:00200000:2.0:1787827241.553238:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.553247:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.553254:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.553262:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.553267:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.553270:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.553273:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.553277:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.553279:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.553285:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.553292:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.553295:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.553297:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.553299:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.553301:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.553309:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub24, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.553311:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.553314:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.553317:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub24, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.553319:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub24, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.553338:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.553344:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d21c0 x1874671476656000/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.553355:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d21c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656000:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.553392:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.554378:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d21c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656000:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.554394:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x6a:0x0](ffff9a789dff42c8) 00000080:00200000:2.0:1787827241.554407:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub24, unhashed 1 00000080:00200000:2.0:1787827241.554424:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.554431:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.554438:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.554445:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.554450:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.554453:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.554455:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.554458:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.554461:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.554467:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.554473:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.554475:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.554477:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.554479:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.554481:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.554488:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub25, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.554490:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.554493:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.554495:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub25, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.554497:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub25, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.554530:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.554536:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d12c0 x1874671476656256/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.554546:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d12c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656256:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.554580:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.555577:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d12c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656256:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.555595:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x6b:0x0](ffff9a789dff7448) 00000080:00200000:2.0:1787827241.555609:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub25, unhashed 1 00000080:00200000:2.0:1787827241.555629:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.555636:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.555643:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.555652:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.555658:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.555661:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.555663:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.555667:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.555669:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.555675:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.555681:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.555684:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.555687:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.555689:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.555691:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.555697:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub26, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.555700:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.555703:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.555705:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub26, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.555707:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub26, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.555726:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.555731:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a4d2940 x1874671476656512/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.555742:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a4d2940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656512:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.555779:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.556662:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a4d2940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656512:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.556677:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x6c:0x0](ffff9a789dff5348) 00000080:00200000:2.0:1787827241.556689:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub26, unhashed 1 00000080:00200000:2.0:1787827241.556706:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.556713:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.556719:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.556726:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.556731:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.556734:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.556736:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.556740:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.556743:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.556749:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.556755:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.556758:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.556760:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.556762:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.556764:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.556770:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub27, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.556773:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.556776:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.556778:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub27, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.556780:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub27, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.556799:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.556804:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d83c0 x1874671476656768/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.556815:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d83c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656768:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.556883:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.557735:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d83c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476656768:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.557750:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x6d:0x0](ffff9a789dff5b88) 00000080:00200000:2.0:1787827241.557762:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub27, unhashed 1 00000080:00200000:2.0:1787827241.557779:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.557785:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.557792:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.557799:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.557804:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.557807:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.557808:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.557812:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.557815:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.557824:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.557856:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.557861:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.557866:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.557869:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.557872:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.557880:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub28, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.557882:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.557885:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.557888:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub28, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.557890:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub28, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.557910:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.557915:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d8000 x1874671476657024/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.557925:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d8000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657024:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.557963:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.558989:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d8000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657024:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.559004:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x6e:0x0](ffff9a789dff2a08) 00000080:00200000:2.0:1787827241.559028:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub28, unhashed 1 00000080:00200000:2.0:1787827241.559044:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.559051:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.559057:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.559064:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.559069:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.559072:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.559074:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.559077:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.559079:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.559085:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.559091:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.559094:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.559097:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.559099:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.559100:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.559106:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub29, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.559108:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.559111:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.559113:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub29, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.559115:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub29, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.559132:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.559137:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d92c0 x1874671476657280/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.559147:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d92c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657280:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.559183:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.560126:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d92c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657280:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.560142:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x6f:0x0](ffff9a789dff1988) 00000080:00200000:2.0:1787827241.560154:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub29, unhashed 1 00000080:00200000:2.0:1787827241.560170:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.560178:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.560186:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.560197:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.560204:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.560209:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.560212:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.560218:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.560222:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.560230:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.560239:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.560242:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.560245:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.560247:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.560248:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.560255:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub30, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.560258:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.560261:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.560263:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub30, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.560265:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub30, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.560284:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.560289:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66dbc00 x1874671476657536/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.560298:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66dbc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657536:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.560330:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.561492:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66dbc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657536:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.561523:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x70:0x0](ffff9a789dff0908) 00000080:00200000:2.0:1787827241.561545:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub30, unhashed 1 00000080:00200000:2.0:1787827241.561561:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.561568:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.561574:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.561581:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.561592:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.561595:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.561597:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.561600:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.561602:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.561608:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.561614:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.561617:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.561619:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.561621:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.561622:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.561628:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub31, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.561630:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.561633:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.561636:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub31, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.561638:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub31, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.561655:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.561660:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d9a40 x1874671476657792/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.561669:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d9a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657792:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.561712:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.562704:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d9a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476657792:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.562724:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x71:0x0](ffff9a789dff63c8) 00000080:00200000:2.0:1787827241.562738:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub31, unhashed 1 00000080:00200000:2.0:1787827241.562761:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.562771:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.562778:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.562788:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.562794:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.562798:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.562801:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.562804:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.562807:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.562816:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.562825:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.562849:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.562854:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.562858:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.562861:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.562869:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub32, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.562873:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.562877:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.562880:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub32, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.562884:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub32, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.562908:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.562913:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d8780 x1874671476658048/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.562923:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d8780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658048:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.562954:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.563949:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d8780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658048:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.563964:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x72:0x0](ffff9a789dff4b08) 00000080:00200000:2.0:1787827241.563975:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub32, unhashed 1 00000080:00200000:2.0:1787827241.563991:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.563997:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.564003:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.564010:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.564015:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.564018:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.564020:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.564023:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.564025:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.564031:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.564038:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.564040:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.564042:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.564044:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.564046:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.564051:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub33, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.564054:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.564057:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.564059:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub33, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.564061:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub33, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.564077:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.564082:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66db0c0 x1874671476658304/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.564092:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66db0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658304:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.564125:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.564972:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66db0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658304:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.564986:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x73:0x0](ffff9a789dff3a88) 00000080:00200000:2.0:1787827241.564997:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub33, unhashed 1 00000080:00200000:2.0:1787827241.565013:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.565020:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.565026:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.565032:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.565038:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.565040:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.565042:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.565045:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.565048:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.565053:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.565059:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.565062:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.565064:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.565066:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.565067:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.565074:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub34, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.565076:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.565079:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.565081:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub34, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.565083:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub34, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.565099:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.565104:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66dad00 x1874671476658560/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.565114:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66dad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658560:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.565145:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.566144:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66dad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658560:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.566160:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x74:0x0](ffff9a789dff1148) 00000080:00200000:2.0:1787827241.566171:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub34, unhashed 1 00000080:00200000:2.0:1787827241.566192:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.566201:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.566210:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.566220:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.566228:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.566233:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.566235:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.566239:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.566241:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.566247:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.566253:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.566256:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.566258:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.566261:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.566262:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.566269:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub35, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.566271:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.566274:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.566276:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub35, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.566278:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub35, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.566296:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.566301:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66da1c0 x1874671476658816/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.566310:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66da1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658816:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.566341:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.567234:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66da1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476658816:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.567248:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x75:0x0](ffff9a789dff00c8) 00000080:00200000:2.0:1787827241.567259:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub35, unhashed 1 00000080:00200000:2.0:1787827241.567275:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.567281:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.567287:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.567294:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.567299:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.567301:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.567304:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.567307:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.567310:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.567315:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.567322:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.567324:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.567326:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.567328:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.567330:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.567336:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub36, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.567338:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.567341:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.567343:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub36, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.567345:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub36, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.567361:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.567366:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d9680 x1874671476659072/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.567376:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d9680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659072:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.567407:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.568399:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d9680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659072:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.568415:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x76:0x0](ffff9a789dff21c8) 00000080:00200000:2.0:1787827241.568426:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub36, unhashed 1 00000080:00200000:2.0:1787827241.568442:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.568448:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.568454:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.568461:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.568466:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.568469:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.568471:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.568474:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.568476:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.568482:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.568488:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.568490:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.568492:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.568494:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.568495:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.568518:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub37, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.568521:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.568525:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.568527:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub37, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.568529:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub37, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.568547:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.568552:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66da940 x1874671476659328/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.568562:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66da940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659328:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.568592:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.569526:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66da940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659328:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.569543:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x77:0x0](ffff9a789dff3248) 00000080:00200000:2.0:1787827241.569555:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub37, unhashed 1 00000080:00200000:2.0:1787827241.569574:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.569582:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.569589:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.569597:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.569603:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.569606:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.569608:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.569612:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.569615:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.569621:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.569627:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.569630:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.569633:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.569636:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.569638:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.569644:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub38, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.569646:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.569649:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.569651:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub38, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.569654:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub38, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.569672:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.569677:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d8f00 x1874671476659584/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.569688:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d8f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659584:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.569725:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.570781:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d8f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659584:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.570810:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x78:0x0](ffff9a789dffe3c8) 00000080:00200000:2.0:1787827241.570822:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub38, unhashed 1 00000080:00200000:2.0:1787827241.570857:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.570865:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.570872:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.570879:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.570884:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.570887:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.570890:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.570893:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.570896:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.570902:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.570908:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.570910:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.570913:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.570914:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.570916:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.570921:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub39, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.570924:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.570927:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.570929:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub39, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.570932:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub39, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.570949:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.570954:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66db840 x1874671476659840/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.570965:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66db840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659840:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.571007:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.571925:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66db840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476659840:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.571940:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x79:0x0](ffff9a789dffcb08) 00000080:00200000:2.0:1787827241.571952:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub39, unhashed 1 00000080:00200000:2.0:1787827241.571967:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.571974:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.571980:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.571987:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.571992:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.571995:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.571997:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.572000:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.572002:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.572008:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.572014:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.572016:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.572019:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.572020:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.572022:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.572028:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub40, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.572030:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.572033:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.572035:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub40, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.572038:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub40, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.572054:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.572059:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d8b40 x1874671476660096/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.572069:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d8b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660096:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.572101:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.573040:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d8b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660096:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.573054:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x7a:0x0](ffff9a789dffba88) 00000080:00200000:2.0:1787827241.573066:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub40, unhashed 1 00000080:00200000:2.0:1787827241.573082:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.573088:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.573094:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.573101:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.573107:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.573110:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.573112:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.573115:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.573117:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.573123:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.573129:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.573140:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.573145:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.573148:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.573150:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.573156:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub41, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.573158:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.573161:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.573163:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub41, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.573165:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub41, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.573183:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.573187:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66da580 x1874671476660352/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.573198:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66da580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660352:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.573228:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.574212:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66da580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660352:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.574232:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x7b:0x0](ffff9a789dff9148) 00000080:00200000:2.0:1787827241.574245:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub41, unhashed 1 00000080:00200000:2.0:1787827241.574263:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.574271:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.574278:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.574286:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.574291:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.574294:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.574296:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.574300:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.574302:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.574308:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.574315:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.574318:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.574320:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.574322:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.574324:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.574331:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub42, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.574333:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.574336:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.574339:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub42, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.574341:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub42, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.574359:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.574365:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66d9e00 x1874671476660608/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.574376:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66d9e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660608:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.574407:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.575336:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66d9e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660608:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.575356:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x7c:0x0](ffff9a789dff80c8) 00000080:00200000:2.0:1787827241.575371:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub42, unhashed 1 00000080:00200000:2.0:1787827241.575391:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.575401:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.575409:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.575416:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.575421:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.575425:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.575427:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.575430:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.575432:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.575438:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.575444:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.575446:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.575449:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.575451:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.575453:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.575458:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub43, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.575460:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.575463:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.575465:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub43, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.575468:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub43, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.575486:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.575491:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ae580 x1874671476660864/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.575525:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ae580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660864:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.575566:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.576475:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ae580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476660864:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.576495:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x7d:0x0](ffff9a789dffa1c8) 00000080:00200000:2.0:1787827241.576525:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub43, unhashed 1 00000080:00200000:2.0:1787827241.576541:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.576547:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.576554:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.576561:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.576566:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.576569:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.576571:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.576574:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.576576:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.576582:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.576588:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.576590:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.576593:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.576595:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.576597:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.576602:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub44, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.576604:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.576607:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.576610:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub44, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.576612:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub44, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.576629:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.576634:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887af840 x1874671476661120/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.576644:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887af840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661120:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.576676:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.577593:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887af840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661120:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.577611:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x7e:0x0](ffff9a789dffb248) 00000080:00200000:2.0:1787827241.577624:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub44, unhashed 1 00000080:00200000:2.0:1787827241.577650:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.577658:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.577665:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.577674:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.577679:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.577683:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.577685:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.577689:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.577691:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.577702:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.577709:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.577713:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.577715:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.577718:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.577720:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.577726:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub45, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.577728:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.577732:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.577734:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub45, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.577736:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub45, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.577755:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.577760:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ae940 x1874671476661376/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.577771:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ae940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661376:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.577810:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.578686:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ae940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661376:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.578701:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x7f:0x0](ffff9a789dffec08) 00000080:00200000:2.0:1787827241.578713:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub45, unhashed 1 00000080:00200000:2.0:1787827241.578730:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.578737:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.578743:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.578750:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.578756:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.578759:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.578761:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.578764:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.578766:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.578772:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.578778:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.578780:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.578783:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.578785:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.578787:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.578792:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub46, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.578795:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.578798:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.578800:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub46, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.578802:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub46, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.578819:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.578824:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ade00 x1874671476661632/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.578853:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ade00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661632:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.578889:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.579794:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ade00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661632:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.579811:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x80:0x0](ffff9a789dffc2c8) 00000080:00200000:2.0:1787827241.579879:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub46, unhashed 1 00000080:00200000:2.0:1787827241.579895:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.579902:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.579908:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.579915:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.579920:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.579924:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.579925:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.579929:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.579931:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.579937:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.579943:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.579945:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.579948:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.579950:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.579952:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.579958:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub47, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.579960:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.579963:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.579965:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub47, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.579968:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub47, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.579985:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.579990:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ac000 x1874671476661888/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.580000:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ac000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661888:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.580043:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.581009:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ac000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476661888:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.581025:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x81:0x0](ffff9a789dfff448) 00000080:00200000:2.0:1787827241.581038:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub47, unhashed 1 00000080:00200000:2.0:1787827241.581055:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.581061:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.581069:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.581077:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.581082:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.581085:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.581088:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.581091:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.581094:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.581100:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.581106:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.581109:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.581111:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.581113:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.581115:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.581121:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub48, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.581124:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.581127:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.581137:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub48, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.581141:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub48, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.581161:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.581166:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ae1c0 x1874671476662144/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.581177:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ae1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662144:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.581211:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.582139:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ae1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662144:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.582155:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x82:0x0](ffff9a789dffd348) 00000080:00200000:2.0:1787827241.582179:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub48, unhashed 1 00000080:00200000:2.0:1787827241.582195:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.582202:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.582209:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.582216:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.582221:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.582224:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.582226:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.582229:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.582232:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.582238:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.582244:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.582247:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.582249:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.582251:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.582253:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.582259:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub49, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.582261:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.582264:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.582266:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub49, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.582268:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub49, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.582286:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.582291:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ad680 x1874671476662400/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.582301:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ad680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662400:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.582335:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.583249:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ad680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662400:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.583264:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x83:0x0](ffff9a789dffdb88) 00000080:00200000:2.0:1787827241.583275:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub49, unhashed 1 00000080:00200000:2.0:1787827241.583291:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.583297:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.583304:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.583311:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.583316:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.583319:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.583321:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.583324:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.583326:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.583332:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.583338:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.583341:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.583343:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.583345:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.583347:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.583353:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub50, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.583355:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.583358:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.583361:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub50, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.583363:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub50, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.583380:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.583385:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ac780 x1874671476662656/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.583395:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ac780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662656:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.583434:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.584315:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ac780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662656:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.584331:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x84:0x0](ffff9a789dffaa08) 00000080:00200000:2.0:1787827241.584343:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub50, unhashed 1 00000080:00200000:2.0:1787827241.584358:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.584365:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.584371:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.584378:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.584383:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.584386:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.584389:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.584392:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.584394:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.584400:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.584406:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.584408:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.584411:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.584414:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.584416:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.584425:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub51, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.584429:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.584433:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.584437:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub51, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.584441:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub51, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.584465:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.584472:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887aed00 x1874671476662912/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.584483:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887aed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662912:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.584530:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.585470:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887aed00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476662912:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.585487:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x85:0x0](ffff9a789dff9988) 00000080:00200000:2.0:1787827241.585499:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub51, unhashed 1 00000080:00200000:2.0:1787827241.585532:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.585539:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.585545:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.585552:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.585557:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.585561:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.585562:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.585565:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.585568:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.585573:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.585579:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.585581:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.585584:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.585585:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.585587:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.585593:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub52, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.585595:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.585598:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.585600:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub52, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.585603:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub52, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.585620:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.585625:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887afc00 x1874671476663168/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.585635:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887afc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663168:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.585666:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.586653:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887afc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663168:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.586668:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x86:0x0](ffff9a789dff8908) 00000080:00200000:2.0:1787827241.586680:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub52, unhashed 1 00000080:00200000:2.0:1787827241.586695:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.586701:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.586707:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.586714:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.586719:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.586722:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.586724:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.586727:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.586729:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.586735:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.586741:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.586743:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.586746:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.586748:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.586750:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.586755:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub53, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.586757:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.586760:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.586762:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub53, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.586764:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub53, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.586780:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.586785:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887acb40 x1874671476663424/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.586795:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887acb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663424:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.586826:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.587714:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887acb40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663424:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.587742:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x87:0x0](ffff9a789dde4b08) 00000080:00200000:2.0:1787827241.587755:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub53, unhashed 1 00000080:00200000:2.0:1787827241.587770:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.587777:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.587783:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.587790:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.587795:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.587797:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.587800:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.587803:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.587806:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.587812:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.587818:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.587820:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.587822:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.587824:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.587826:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.587846:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub54, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.587849:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.587852:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.587855:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub54, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.587857:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub54, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.587874:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.587880:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78887ac3c0 x1874671476663680/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.587889:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78887ac3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663680:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.587922:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.588849:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78887ac3c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663680:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.588864:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x88:0x0](ffff9a789dde3a88) 00000080:00200000:2.0:1787827241.588876:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub54, unhashed 1 00000080:00200000:2.0:1787827241.588892:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.588899:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.588905:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.588912:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.588917:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.588920:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.588922:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.588925:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.588927:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.588933:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.588939:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.588950:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.588952:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.588954:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.588956:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.588962:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub55, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.588964:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.588968:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.588970:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub55, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.588972:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub55, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.589001:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.589006:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665bc00 x1874671476663936/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.589016:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665bc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663936:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.589057:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.589979:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665bc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476663936:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.589997:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x89:0x0](ffff9a789dde1148) 00000080:00200000:2.0:1787827241.590010:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub55, unhashed 1 00000080:00200000:2.0:1787827241.590027:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.590034:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.590041:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.590050:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.590055:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.590059:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.590061:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.590064:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.590067:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.590073:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.590079:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.590082:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.590085:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.590087:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.590089:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.590096:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub56, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.590099:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.590102:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.590104:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub56, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.590106:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub56, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.590124:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.590129:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665b480 x1874671476664192/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.590140:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665b480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476664192:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.590175:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.591131:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665b480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476664192:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.591152:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x8a:0x0](ffff9a789dde00c8) 00000080:00200000:2.0:1787827241.591168:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub56, unhashed 1 00000080:00200000:2.0:1787827241.591190:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.591200:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.591209:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.591220:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.591228:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.591234:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.591237:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.591243:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.591247:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.591256:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.591267:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.591271:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.591276:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.591279:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.591282:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.591292:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub57, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.591296:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.591301:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.591305:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub57, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.591308:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub57, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.591334:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.591341:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6659e00 x1874671476664448/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.591357:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6659e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476664448:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.591405:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.592414:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6659e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476664448:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.592436:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x8b:0x0](ffff9a789dde21c8) 00000080:00200000:2.0:1787827241.592450:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub57, unhashed 1 00000080:00200000:2.0:1787827241.592466:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.592474:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.592480:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.592488:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.592493:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.592496:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.592497:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.592517:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.592521:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.592528:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.592535:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.592538:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.592541:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.592543:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.592544:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.592551:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub58, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.592553:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.592557:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.592559:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub58, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.592561:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub58, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.592580:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.592585:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665a580 x1874671476664704/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.592596:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665a580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476664704:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.592629:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000080:00200000:3.0:1787827241.602091:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.602095:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.602097:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.602099:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.602105:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub66, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.602107:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.602110:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.602112:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub66, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.602114:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub66, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.602133:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.602138:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78838c8780 x1874671476666752/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.602148:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c8780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476666752:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.602190:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.603225:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c8780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476666752:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.603243:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x94:0x0](ffff9a789ddb00c8) 00000080:00200000:3.0:1787827241.603260:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub66, unhashed 1 00000080:00200000:3.0:1787827241.603282:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.603291:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.603300:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.603311:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.603318:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.603323:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.603326:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.603332:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.603336:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.603345:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.603355:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.603358:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.603361:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.603363:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.603365:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.603371:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub67, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.603373:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.603376:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.603379:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub67, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.603381:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub67, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.603400:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.603405:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78838cb480 x1874671476667008/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.603419:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838cb480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667008:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.603461:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.604457:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838cb480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667008:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.604473:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x95:0x0](ffff9a789dfdb248) 00000080:00200000:3.0:1787827241.604484:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub67, unhashed 1 00000080:00200000:3.0:1787827241.604516:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.604525:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.604534:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.604545:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.604553:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.604558:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.604561:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.604565:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.604569:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.604577:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.604584:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.604588:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.604590:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.604593:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.604595:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.604601:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub68, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.604603:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.604606:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.604609:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub68, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.604611:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub68, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.604630:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.604636:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78838ca940 x1874671476667264/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.604646:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838ca940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667264:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.604678:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.605659:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838ca940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667264:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.605675:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x96:0x0](ffff9a789dfda1c8) 00000080:00200000:3.0:1787827241.605687:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub68, unhashed 1 00000080:00200000:3.0:1787827241.605703:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.605710:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.605716:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.605723:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.605728:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.605731:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.605733:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.605737:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.605739:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.605745:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.605751:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.605753:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.605756:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.605758:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.605760:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.605775:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub69, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.605777:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.605780:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.605783:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub69, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.605785:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub69, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.605803:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.605808:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78838cad00 x1874671476667520/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.605818:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838cad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667520:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.605915:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.606850:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838cad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667520:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.606872:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x97:0x0](ffff9a789dfdaa08) 00000080:00200000:3.0:1787827241.606885:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub69, unhashed 1 00000080:00200000:3.0:1787827241.606901:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.606907:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.606914:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.606921:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.606926:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.606929:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.606931:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.606934:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.606936:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.606942:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.606949:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.606951:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.606953:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.606955:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.606957:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.606962:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub70, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.606964:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.606967:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.606969:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub70, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.606971:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub70, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.606988:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.606993:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78838c9680 x1874671476667776/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.607004:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c9680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667776:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.607037:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.608347:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c9680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476667776:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.608365:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x98:0x0](ffff9a789dfd8908) 00000080:00200000:3.0:1787827241.608380:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub70, unhashed 1 00000080:00200000:3.0:1787827241.608399:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.608407:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.608416:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.608428:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.608437:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.608442:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.608445:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.608450:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.608453:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.608463:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.608475:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.608479:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.608483:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.608487:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.608490:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.608497:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub71, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.608509:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.608515:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.608518:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub71, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.608521:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub71, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.608543:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.608548:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78838ca580 x1874671476668032/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.608560:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838ca580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668032:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.608594:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.609694:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838ca580 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668032:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.609711:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x99:0x0](ffff9a789dfdd348) 00000080:00200000:3.0:1787827241.609724:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub71, unhashed 1 00000080:00200000:3.0:1787827241.609741:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.609748:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.609755:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.609762:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.609767:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.609770:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.609772:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.609776:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.609778:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.609784:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.609790:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.609793:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.609796:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.609798:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.609800:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.609806:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub72, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.609808:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.609811:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.609814:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub72, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.609816:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub72, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.609885:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.609893:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78838c8f00 x1874671476668288/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.609905:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78838c8f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668288:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.609957:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.610987:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78838c8f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668288:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.611004:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x9a:0x0](ffff9a789dfde3c8) 00000080:00200000:3.0:1787827241.611017:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub72, unhashed 1 00000080:00200000:3.0:1787827241.611038:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.611047:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.611057:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.611067:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.611075:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.611079:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.611083:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.611088:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.611092:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.611101:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.611111:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.611116:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.611120:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.611124:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.611127:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.611136:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub73, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.611139:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.611144:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.611148:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub73, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.611151:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub73, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.611188:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.611195:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a788a57f840 x1874671476668544/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.611212:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a788a57f840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668544:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.611249:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.612349:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a788a57f840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668544:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.612366:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x9b:0x0](ffff9a789dfdec08) 00000080:00200000:3.0:1787827241.612380:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub73, unhashed 1 00000080:00200000:3.0:1787827241.612399:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.612407:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.612414:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.612423:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.612428:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.612432:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.612434:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.612437:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.612440:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.612446:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.612453:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.612456:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.612459:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.612461:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.612463:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.612468:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub74, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.612471:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.612474:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.612477:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub74, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.612479:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub74, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.612499:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.612520:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66f9a40 x1874671476668800/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.612532:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f9a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668800:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.612569:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.613680:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66f9a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476668800:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.613703:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x9c:0x0](ffff9a789dfdcb08) 00000080:00200000:3.0:1787827241.613718:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub74, unhashed 1 00000080:00200000:3.0:1787827241.613736:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.613743:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.613749:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.613757:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.613762:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.613766:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.613768:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.613771:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.613774:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.613780:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.613786:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.613789:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.613792:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.613794:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.613796:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.613802:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub75, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.613804:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.613807:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.613809:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub75, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.613812:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub75, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.613859:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.613864:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66fb0c0 x1874671476669056/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.613876:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fb0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669056:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.613921:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.614975:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66fb0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669056:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.614991:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x9d:0x0](ffff9a789dfdc2c8) 00000080:00200000:3.0:1787827241.615003:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub75, unhashed 1 00000080:00200000:3.0:1787827241.615022:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.615028:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.615035:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.615043:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.615048:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.615051:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.615053:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.615057:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.615059:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.615065:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.615072:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.615075:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.615077:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.615079:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.615081:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.615086:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub76, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.615089:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.615091:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.615094:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub76, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.615096:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub76, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.615114:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.615120:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66f9e00 x1874671476669312/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.615138:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f9e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669312:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.615183:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.616486:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66f9e00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669312:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.616527:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x9e:0x0](ffff9a789dfd80c8) 00000080:00200000:3.0:1787827241.616546:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub76, unhashed 1 00000080:00200000:3.0:1787827241.616572:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.616582:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.616592:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.616604:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.616613:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.616618:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.616621:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.616625:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.616629:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.616638:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.616648:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.616653:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.616657:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.616661:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.616664:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.616674:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub77, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.616677:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.616682:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.616685:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub77, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.616689:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub77, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.616717:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.616724:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66fa940 x1874671476669568/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.616742:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fa940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669568:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.616811:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.617909:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66fa940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669568:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.617929:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0x9f:0x0](ffff9a789dfdf448) 00000080:00200000:3.0:1787827241.617946:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub77, unhashed 1 00000080:00200000:3.0:1787827241.617971:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.617981:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.617990:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.618001:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.618009:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.618013:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.618015:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.618019:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.618022:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.618030:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.618039:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.618043:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.618048:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.618051:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.618053:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.618062:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub78, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.618065:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.618069:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.618072:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub78, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.618075:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub78, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.618099:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.618105:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66f8f00 x1874671476669824/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.618119:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f8f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669824:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.618160:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.619146:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66f8f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476669824:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.619165:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa0:0x0](ffff9a789dfd9148) 00000080:00200000:3.0:1787827241.619181:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub78, unhashed 1 00000080:00200000:3.0:1787827241.619205:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.619214:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.619228:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.619239:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.619247:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.619251:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.619255:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.619260:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.619264:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.619272:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.619282:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.619287:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.619291:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.619295:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.619298:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.619306:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub79, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.619310:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.619314:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.619318:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub79, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.619321:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub79, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.619345:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.619353:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66f92c0 x1874671476670080/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.619367:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f92c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670080:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.619414:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.620565:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66f92c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670080:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.620589:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa1:0x0](ffff9a789dfdba88) 00000080:00200000:3.0:1787827241.620607:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub79, unhashed 1 00000080:00200000:3.0:1787827241.620634:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.620644:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.620653:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.620664:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.620672:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.620677:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.620680:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.620685:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.620689:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.620697:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.620708:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.620713:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.620717:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.620720:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.620724:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.620733:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub80, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.620737:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.620741:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.620744:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub80, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.620748:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub80, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.620774:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.620781:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66f83c0 x1874671476670336/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.620798:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f83c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670336:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.620875:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.622016:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66f83c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670336:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.622039:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa2:0x0](ffff9a789dfd9988) 00000080:00200000:3.0:1787827241.622057:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub80, unhashed 1 00000080:00200000:3.0:1787827241.622080:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.622089:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.622099:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.622110:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.622119:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.622124:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.622127:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.622131:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.622135:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.622144:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.622154:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.622159:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.622164:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.622167:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.622170:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.622179:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub81, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.622183:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.622187:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.622191:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub81, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.622194:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub81, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.622218:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.622226:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66fb480 x1874671476670592/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.622242:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fb480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670592:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.622284:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.623767:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66fb480 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670592:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.623805:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa3:0x0](ffff9a789dfddb88) 00000080:00200000:3.0:1787827241.623861:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub81, unhashed 1 00000080:00200000:3.0:1787827241.623902:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.623924:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.623940:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.623954:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.623963:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.623969:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.623972:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.623980:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.623984:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.623993:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.624007:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.624013:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.624018:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.624022:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.624025:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.624037:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub82, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.624041:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.624046:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.624049:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub82, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.624053:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub82, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.624099:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.624109:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66fbc00 x1874671476670848/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.624136:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fbc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670848:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.624229:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787827241.625922:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66fbc00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476670848:192.168.203.140@tcp:36:createmany.0 00000080:00200000:3.0:1787827241.625951:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa4:0x0](ffff9a789dfd0908) 00000080:00200000:3.0:1787827241.625979:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub82, unhashed 1 00000080:00200000:3.0:1787827241.626016:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.626028:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.626040:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.626050:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.626056:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.626060:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.626062:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.626066:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:3.0:1787827241.626069:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.626076:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.626084:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.626088:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:3.0:1787827241.626091:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.626093:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:3.0:1787827241.626095:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:3.0:1787827241.626102:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub83, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:3.0:1787827241.626104:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.626107:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:3.0:1787827241.626110:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub83, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:3.0:1787827241.626112:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub83, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:3.0:1787827241.626151:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787827241.626157:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66f8000 x1874671476671104/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827241.626172:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f8000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671104:192.168.203.140@tcp:36:createmany.0 00000100:00100000:3.0:1787827241.626217:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a7887648cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.627541:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66f8000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671104:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.627580:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa5:0x0](ffff9a789dde3248) 00000080:00200000:2.0:1787827241.627607:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub83, unhashed 1 00000080:00200000:2.0:1787827241.627645:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.627658:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.627672:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.627685:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.627693:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.627700:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.627703:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.627708:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.627712:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.627722:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.627735:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.627741:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.627746:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.627750:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.627753:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.627764:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub84, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.627768:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.627775:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.627779:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub84, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.627782:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub84, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.627820:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.627852:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6658b40 x1874671476671360/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.627875:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6658b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671360:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.627951:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.629541:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6658b40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671360:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.629581:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa6:0x0](ffff9a789dde6c08) 00000080:00200000:2.0:1787827241.629617:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub84, unhashed 1 00000080:00200000:2.0:1787827241.629663:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.629676:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.629690:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.629702:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.629708:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.629713:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.629715:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.629721:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.629724:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.629731:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.629741:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.629746:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.629749:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.629751:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.629754:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.629762:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub85, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.629764:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.629769:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.629772:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub85, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.629774:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub85, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.629811:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.629817:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665b840 x1874671476671616/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.629865:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665b840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671616:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.629937:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.631065:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665b840 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671616:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.631087:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa7:0x0](ffff9a789dde42c8) 00000080:00200000:2.0:1787827241.631103:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub85, unhashed 1 00000080:00200000:2.0:1787827241.631134:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.631146:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.631158:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.631171:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.631180:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.631185:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.631188:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.631193:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.631195:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.631202:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.631209:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.631213:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.631216:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.631218:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.631221:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.631228:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub86, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.631230:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.631233:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.631236:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub86, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.631238:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub86, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.631259:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.631265:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665a940 x1874671476671872/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.631287:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665a940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671872:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.631330:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.632481:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665a940 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476671872:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.632515:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa8:0x0](ffff9a789dde7448) 00000080:00200000:2.0:1787827241.632541:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub86, unhashed 1 00000080:00200000:2.0:1787827241.632565:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.632573:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.632584:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.632593:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.632599:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.632603:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.632605:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.632608:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.632611:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.632617:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.632625:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.632628:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.632632:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.632634:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.632636:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.632643:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub87, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.632645:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.632649:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.632652:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub87, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.632654:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub87, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.632678:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.632683:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6659680 x1874671476672128/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.632695:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6659680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672128:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.632736:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.633897:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6659680 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672128:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.633922:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xa9:0x0](ffff9a789dde5348) 00000080:00200000:2.0:1787827241.633959:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub87, unhashed 1 00000080:00200000:2.0:1787827241.633991:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.634003:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.634015:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.634027:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.634034:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.634038:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.634040:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.634045:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.634048:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.634057:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.634069:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.634075:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.634079:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.634083:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.634086:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.634096:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub88, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.634099:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.634105:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.634118:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub88, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.634121:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub88, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.634146:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.634152:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6658f00 x1874671476672384/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.634164:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6658f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672384:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.634231:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.635488:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6658f00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672384:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.635522:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xaa:0x0](ffff9a789dde5b88) 00000080:00200000:2.0:1787827241.635539:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub88, unhashed 1 00000080:00200000:2.0:1787827241.635571:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.635580:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.635592:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.635601:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.635607:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.635610:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.635612:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.635615:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.635617:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.635624:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.635631:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.635634:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.635637:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.635638:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.635640:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.635648:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub89, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.635651:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.635654:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.635657:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub89, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.635659:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub89, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.635687:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.635695:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665ad00 x1874671476672640/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.635712:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665ad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672640:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.635757:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.637110:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665ad00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672640:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.637129:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xab:0x0](ffff9a789dde2a08) 00000080:00200000:2.0:1787827241.637143:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub89, unhashed 1 00000080:00200000:2.0:1787827241.637161:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.637168:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.637176:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.637184:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.637190:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.637193:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.637195:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.637198:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.637200:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.637207:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.637213:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.637216:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.637218:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.637220:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.637223:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.637229:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub90, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.637231:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.637234:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.637237:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub90, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.637239:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub90, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.637257:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.637262:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665b0c0 x1874671476672896/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.637273:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665b0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672896:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.637310:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.638402:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665b0c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476672896:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.638422:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xac:0x0](ffff9a789dde1988) 00000080:00200000:2.0:1787827241.638439:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub90, unhashed 1 00000080:00200000:2.0:1787827241.638461:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.638471:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.638480:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.638490:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.638496:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.638514:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.638517:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.638522:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.638524:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.638530:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.638536:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.638539:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.638542:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.638544:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.638545:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.638552:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub91, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.638554:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.638558:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.638560:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub91, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.638562:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub91, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.638581:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.638586:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a665a1c0 x1874671476673152/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.638596:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a665a1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673152:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.638629:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.640124:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a665a1c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673152:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.640160:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xad:0x0](ffff9a789dde0908) 00000080:00200000:2.0:1787827241.640193:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub91, unhashed 1 00000080:00200000:2.0:1787827241.640228:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.640242:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.640257:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.640270:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.640278:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.640283:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.640285:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.640291:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.640294:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.640303:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.640316:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.640321:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.640326:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.640330:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.640333:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.640345:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub92, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.640348:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.640353:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.640357:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub92, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.640361:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub92, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.640403:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.640411:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66592c0 x1874671476673408/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.640431:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66592c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673408:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.640515:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.642267:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66592c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673408:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.642304:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xae:0x0](ffff9a789dde63c8) 00000080:00200000:2.0:1787827241.642330:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub92, unhashed 1 00000080:00200000:2.0:1787827241.642373:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.642389:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.642406:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.642420:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.642426:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.642431:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.642433:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.642439:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.642441:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.642448:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.642458:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.642462:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.642465:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.642468:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.642470:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.642480:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub93, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.642482:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.642486:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.642489:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub93, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.642492:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub93, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.642553:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.642559:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6658780 x1874671476673664/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.642574:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6658780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673664:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.642632:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.643811:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6658780 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673664:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.643895:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xaf:0x0](ffff9a789ddea1c8) 00000080:00200000:2.0:1787827241.643931:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub93, unhashed 1 00000080:00200000:2.0:1787827241.643962:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.643973:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.643984:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.643998:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.644007:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.644012:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.644016:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.644022:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.644026:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.644036:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.644048:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.644053:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.644058:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.644062:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.644065:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.644076:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub94, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.644080:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.644085:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.644089:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub94, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.644093:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub94, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.644123:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.644131:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6659a40 x1874671476673920/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.644148:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6659a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673920:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.644196:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.645423:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6659a40 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476673920:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.645444:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xb0:0x0](ffff9a789ddeb248) 00000080:00200000:2.0:1787827241.645460:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub94, unhashed 1 00000080:00200000:2.0:1787827241.645485:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.645493:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.645517:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.645527:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.645533:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.645537:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.645540:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.645543:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.645546:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.645552:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.645559:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.645562:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.645569:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.645571:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.645573:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.645580:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub95, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.645582:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.645586:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.645589:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub95, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.645592:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub95, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.645615:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.645621:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a6658000 x1874671476674176/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.645638:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6658000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674176:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.645693:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.646824:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6658000 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674176:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.646873:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xb1:0x0](ffff9a789ddeec08) 00000080:00200000:2.0:1787827241.646890:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub95, unhashed 1 00000080:00200000:2.0:1787827241.646912:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.646921:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.646931:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.646942:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.646950:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.646968:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.646971:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.646975:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.646979:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.646987:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.646996:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.647000:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.647005:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.647009:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.647012:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.647020:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub96, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.647023:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.647027:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.647030:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub96, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.647033:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub96, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.647058:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.647064:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a66583c0 x1874671476674432/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.647078:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66583c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674432:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.647137:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.648134:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66583c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674432:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.648153:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xb2:0x0](ffff9a789ddec2c8) 00000080:00200000:2.0:1787827241.648169:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub96, unhashed 1 00000080:00200000:2.0:1787827241.648188:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.648196:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.648205:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.648214:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.648230:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.648234:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.648237:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.648241:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.648245:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.648253:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.648260:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.648264:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.648268:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.648271:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.648273:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.648281:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub97, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.648284:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.648288:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.648291:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub97, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.648294:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub97, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.648328:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.648335:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65f6d00 x1874671476674688/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.648348:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f6d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674688:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.648388:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.649406:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f6d00 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674688:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.649427:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xb3:0x0](ffff9a789ddef448) 00000080:00200000:2.0:1787827241.649443:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub97, unhashed 1 00000080:00200000:2.0:1787827241.649462:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.649470:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.649478:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.649487:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.649494:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.649499:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.649516:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.649520:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.649524:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.649531:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.649540:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.649544:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.649548:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.649551:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.649553:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.649561:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub98, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.649564:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.649569:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.649572:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub98, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.649575:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub98, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.649597:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.649604:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65f70c0 x1874671476674944/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.649617:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f70c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674944:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.649655:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.650547:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f70c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476674944:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.650564:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xb4:0x0](ffff9a789dded348) 00000080:00200000:2.0:1787827241.650579:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub98, unhashed 1 00000080:00200000:2.0:1787827241.650597:0:48389:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:2.0:1787827241.650604:0:48389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787827241.650612:0:48389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.650621:0:48389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.650627:0:48389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787827241.650631:0:48389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:2.0:1787827241.650634:0:48389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:2.0:1787827241.650638:0:48389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bbd0 00010000:00010000:2.0:1787827241.650641:0:48389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787827241.650648:0:48389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.650656:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.650659:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d6.sanity-scrub, flags=24 00000080:00200000:2.0:1787827241.650663:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x200001b72:0x1:0x0](ffff9a789ddb1988), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.650666:0:48389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=mds2, flags=24 00000080:00200000:2.0:1787827241.650669:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 1 00000080:00200000:2.0:1787827241.650676:0:48389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f6.sanity-scrub99, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=1536 00000080:00200000:2.0:1787827241.650680:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.650684:0:48389:0:(file.c:6689:ll_inode_permission()) VFS Op:inode=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), inode mode 41ed mask 3 00000080:00200000:2.0:1787827241.650687:0:48389:0:(namei.c:2129:ll_create_nd()) VFS Op:name=f6.sanity-scrub99, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08), flags=33060, excl=1 00000080:00200000:2.0:1787827241.650689:0:48389:0:(namei.c:2084:ll_mknod()) VFS Op:name=f6.sanity-scrub99, dir=[0x240001b71:0x1:0x0](ffff9a789dfe4b08) mode 100444 dev 0 00000020:00100000:2.0:1787827241.650711:0:48389:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9a78a6512000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787827241.650717:0:48389:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9a78a65f61c0 x1874671476675200/t0(0) o36->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827241.650729:0:48389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f61c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476675200:192.168.203.140@tcp:36:createmany.0 00000100:00100000:2.0:1787827241.650769:0:48389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9a78894aea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787827241.651854:0:48389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f61c0 pname:cluuid:pid:xid:nid:opc:job createmany:97196825-2cb9-49ba-a47d-1455a997ce3a:48389:1874671476675200:192.168.203.140@tcp:36:createmany.0 00000080:00200000:2.0:1787827241.651878:0:48389:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001b71:0xb5:0x0](ffff9a789ddedb88) 00000080:00200000:2.0:1787827241.651894:0:48389:0:(namei.c:2137:ll_create_nd()) VFS Op:name=f6.sanity-scrub99, unhashed 1 00000080:00200008:3.0:1787827241.760387:0:48427:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9a7891728908), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787827241.760398:0:48427:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:3.0:1787827241.760423:0:48427:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787827241.760443:0:48427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.760457:0:48427:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787827241.760464:0:48427:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787827241.760472:0:48427:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:3.0:1787827241.760475:0:48427:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:3.0:1787827241.760483:0:48427:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487e8bcb8 00010000:00010000:3.0:1787827241.760486:0:48427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787827241.760493:0:48427:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200008:1.0:1787827241.957991:0:48428:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9a7891728908), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827241.958004:0:48428:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827241.958028:0:48428:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827241.958048:0:48428:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827241.958063:0:48428:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827241.958071:0:48428:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827241.958078:0:48428:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827241.958081:0:48428:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827241.958089:0:48428:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487dcbcb8 00010000:00010000:1.0:1787827241.958092:0:48428:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827241.958100:0:48428:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200008:1.0:1787827241.958200:0:48428:0:(file.c:6313:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9a7891728908), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787827241.958203:0:48428:0:(file.c:6208:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9a7891728908),name=/ 00000002:00010000:1.0:1787827241.958209:0:48428:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787827241.958215:0:48428:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827241.958222:0:48428:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787827241.958227:0:48428:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787827241.958230:0:48428:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) 00000080:00010000:1.0:1787827241.958232:0:48428:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9a7891728908) for lock 0xebdcd48a4298d99f 00000080:00010000:1.0:1787827241.958235:0:48428:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xebdcd48a4298d99f from it ffffaa5487dcbcb8 00010000:00010000:1.0:1787827241.958237:0:48428:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787827241.958247:0:48428:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bcc00/0xebdcd48a4298d99f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x4059aa0aef8d8ef0 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:2.0:1787827241.967096:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.967113:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.967154:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000067a96828 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000083301894/0xebdcd48a4298d9fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9024 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.967172:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000067a96828 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000083301894/0xebdcd48a4298d9fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9024 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.967224:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000083301894/0xebdcd48a4298d9fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9024 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00000008:00100000:2.0:1787827241.967311:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f5a40 - object 0x280000401:98 offset 0<>903 00000080:00200000:2.0:1787827241.967351:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.967360:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.967364:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.967373:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000003e0fe000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007e3c7e26/0xebdcd48a4298da24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d909b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 1 00000100:00100000:0.0:1787827241.967379:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476675328:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.967382:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000003e0fe000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007e3c7e26/0xebdcd48a4298da24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d909b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 1 00010000:00010000:2.0:1787827241.967408:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007e3c7e26/0xebdcd48a4298da24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d909b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00000008:00100000:2.0:1787827241.967445:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f4000 - object 0x2c0000401:98 offset 0<>5634 00000080:00200000:2.0:1787827241.967482:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.967491:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.967495:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1787827241.967515:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476675456:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.967520:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000054ea9985 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000b250d178/0xebdcd48a4298da47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->393215) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d90f6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.967529:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000054ea9985 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000b250d178/0xebdcd48a4298da47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->393215) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d90f6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.967645:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000b250d178/0xebdcd48a4298da47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->393215) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d90f6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.968122:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f43c0 - object 0x280000401:99 offset 0<>391609 00000080:00200000:2.0:1787827241.968149:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.968157:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.968160:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.968169:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000a8520cc0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e87b1566/0xebdcd48a4298da6a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9151 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.968178:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000a8520cc0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e87b1566/0xebdcd48a4298da6a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9151 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787827241.968184:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1787827241.968199:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e87b1566/0xebdcd48a4298da6a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9151 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00000100:00100000:1.0:1787827241.968200:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476675584:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.968235:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f7c00 - object 0x2c0000401:99 offset 0<>1735 00000080:00200000:2.0:1787827241.968263:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.968271:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.968275:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:3.0:1787827241.968284:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476675712:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.968285:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000004ed82aaa ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bad438eb/0xebdcd48a4298da8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d91ac expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.968299:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000004ed82aaa ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bad438eb/0xebdcd48a4298da8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d91ac expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.968323:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bad438eb/0xebdcd48a4298da8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d91ac expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.968373:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f7480 - object 0x280000401:100 offset 0<>9974 00000080:00200000:2.0:1787827241.968396:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.968407:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.968411:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.968421:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000004ed475b ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007a77bb05/0xebdcd48a4298dab0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9207 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.968429:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476675840:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.968435:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000004ed475b ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007a77bb05/0xebdcd48a4298dab0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9207 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.968480:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007a77bb05/0xebdcd48a4298dab0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9207 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.968540:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f5e00 - object 0x2c0000401:100 offset 0<>38517 00000080:00200000:2.0:1787827241.968562:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.968570:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.968575:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:3.0:1787827241.968575:0:10935:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000008:00010000:2.0:1787827241.968586:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000693bd8f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c827a304/0xebdcd48a4298dad3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9262 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.968596:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000693bd8f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c827a304/0xebdcd48a4298dad3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9262 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.968621:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c827a304/0xebdcd48a4298dad3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9262 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:3.0:1787827241.968634:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476675968:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.968669:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f6580 - object 0x280000401:101 offset 0<>39343 00000100:00100000:3.0:1787827241.968685:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476676096:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.968695:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.968706:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.968711:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.968722:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000003542bbd0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b342868b/0xebdcd48a4298daf6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d92bd expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.968736:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000003542bbd0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b342868b/0xebdcd48a4298daf6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d92bd expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.968767:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b342868b/0xebdcd48a4298daf6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d92bd expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.968809:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f4b40 - object 0x2c0000401:101 offset 0<>15476 00000080:00200000:2.0:1787827241.968859:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:0.0:1787827241.968871:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000080:00200000:2.0:1787827241.968872:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0xb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.968880:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.968888:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476676224:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.968889:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000d5a5292a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000abb2908a/0xebdcd48a4298db19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9318 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.968905:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000d5a5292a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000abb2908a/0xebdcd48a4298db19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9318 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.968932:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000abb2908a/0xebdcd48a4298db19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9318 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.968972:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f7840 - object 0x280000401:102 offset 0<>674 00000080:00200000:2.0:1787827241.969001:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0xb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969016:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:0.0:1787827241.969021:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476676352:192.168.203.140@tcp:4:cp.0 00020000:00200000:2.0:1787827241.969023:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.969037:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000000af7b30c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000eeaaa337/0xebdcd48a4298db3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9373 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.969050:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000000af7b30c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000eeaaa337/0xebdcd48a4298db3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9373 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.969076:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000eeaaa337/0xebdcd48a4298db3c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9373 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.969119:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f6940 - object 0x2c0000401:102 offset 0<>9198 00000080:00200000:2.0:1787827241.969139:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969151:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0xd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:3.0:1787827241.969153:0:10935:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00020000:00200000:2.0:1787827241.969155:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.969165:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000004df7eb6d ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000514aa605/0xebdcd48a4298db5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d93ce expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787827241.969168:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476676480:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.969182:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000004df7eb6d ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000514aa605/0xebdcd48a4298db5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d93ce expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.969202:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000514aa605/0xebdcd48a4298db5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d93ce expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.969234:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f5680 - object 0x280000401:103 offset 0<>2659 00000080:00200000:2.0:1787827241.969254:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0xd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969261:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0xe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.969264:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.969271:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000001d8182b7 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000985be67/0xebdcd48a4298db82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9429 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.969279:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000001d8182b7 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000985be67/0xebdcd48a4298db82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9429 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.969285:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1787827241.969299:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000985be67/0xebdcd48a4298db82 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9429 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.969306:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476676608:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.969328:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f4f00 - object 0x2c0000401:103 offset 0<>4207 00000080:00200000:2.0:1787827241.969352:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0xe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969361:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0xf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.969365:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.969367:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000008:00010000:2.0:1787827241.969374:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000066c4d22f ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000f96f35de/0xebdcd48a4298dba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9484 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.969380:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476676736:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.969387:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000066c4d22f ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000f96f35de/0xebdcd48a4298dba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9484 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.969413:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000f96f35de/0xebdcd48a4298dba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9484 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.969470:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f4780 - object 0x280000401:104 offset 0<>28245 00000080:00200000:2.0:1787827241.969493:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0xf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969516:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x10:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:3.0:1787827241.969519:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00020000:00200000:2.0:1787827241.969521:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:3.0:1787827241.969531:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476676864:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.969532:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000006ecad00a ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c1f046c4/0xebdcd48a4298dbc8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d94df expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.969545:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000006ecad00a ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c1f046c4/0xebdcd48a4298dbc8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d94df expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.969565:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c1f046c4/0xebdcd48a4298dbc8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d94df expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.969589:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000003e0fe000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007e3c7e26/0xebdcd48a4298da24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d909b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00000008:00100000:2.0:1787827241.969592:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a65f52c0 - object 0x2c0000401:104 offset 0<>704 00000080:00200000:2.0:1787827241.969603:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x10:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969611:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x11:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.969614:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.969620:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000043e2d6d1 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000008b9587d9/0xebdcd48a4298dbeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d953a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.969629:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000043e2d6d1 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000008b9587d9/0xebdcd48a4298dbeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d953a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.969631:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000003e0fe000 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007e3c7e26/0xebdcd48a4298da24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d909b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787827241.969653:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000008b9587d9/0xebdcd48a4298dbeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d953a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.969668:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476675456:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.969698:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce1e00 - object 0x280000401:105 offset 0<>1754 00000100:00100000:0.0:1787827241.969699:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a65f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476676992:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.969720:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x11:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969732:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x12:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.969738:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.969741:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000008:00010000:2.0:1787827241.969750:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000cfa2d7fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c87fa88a/0xebdcd48a4298dc0e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9595 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.969753:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476677120:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.969769:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000cfa2d7fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c87fa88a/0xebdcd48a4298dc0e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9595 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.969793:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c87fa88a/0xebdcd48a4298dc0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9595 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.969859:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce2580 - object 0x2c0000401:105 offset 0<>4915 00000080:00200000:2.0:1787827241.969889:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x12:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.969903:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x13:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.969910:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.969913:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476677248:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.969924:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000009068126c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054c2b6dc/0xebdcd48a4298dc31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d95f0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.969938:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000009068126c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054c2b6dc/0xebdcd48a4298dc31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d95f0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.969968:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054c2b6dc/0xebdcd48a4298dc31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d95f0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.970011:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce0b40 - object 0x280000401:106 offset 0<>1613 00000080:00200000:2.0:1787827241.970040:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x13:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970053:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x14:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:0.0:1787827241.970056:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476677376:192.168.203.140@tcp:4:cp.0 00020000:00200000:2.0:1787827241.970061:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.970071:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000a4fc51fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000005bf881b/0xebdcd48a4298dc54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d964b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.970080:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000a4fc51fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000005bf881b/0xebdcd48a4298dc54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d964b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.970103:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000005bf881b/0xebdcd48a4298dc54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d964b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.970142:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce3840 - object 0x2c0000401:106 offset 0<>16105 00000080:00200000:2.0:1787827241.970163:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x14:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970171:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x15:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970176:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.970180:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476677504:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.970184:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x15:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970191:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x16:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970195:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.970201:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000715e91d9 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d155b573/0xebdcd48a4298dc9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9701 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.970208:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000715e91d9 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d155b573/0xebdcd48a4298dc9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9701 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.970236:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d155b573/0xebdcd48a4298dc9a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x6b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9701 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.970272:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce2940 - object 0x2c0000401:107 offset 0<>2858 00000080:00200000:2.0:1787827241.970293:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x16:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970301:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x17:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970305:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.970310:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000080:00200000:2.0:1787827241.970312:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x17:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970320:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x18:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:0.0:1787827241.970323:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476677632:192.168.203.140@tcp:4:cp.0 00020000:00200000:2.0:1787827241.970324:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970333:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x18:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970341:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x19:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970344:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970351:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x19:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970358:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x1a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970361:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970369:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x1a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970377:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x1b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970380:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970386:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x1b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970390:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x1c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970392:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970396:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x1c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970400:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x1d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970402:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970407:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x1d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970411:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x1e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970413:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970416:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x1e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970421:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x1f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970425:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970432:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x1f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970439:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x20:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970443:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970451:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x20:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970458:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x21:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970462:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970471:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x21:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970480:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x22:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970485:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970492:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x22:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970509:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x23:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970514:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970522:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x23:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970527:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x24:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970531:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970536:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x24:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970541:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x25:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970544:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970548:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x25:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970552:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x26:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970554:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970558:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x26:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970562:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x27:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970565:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970569:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x27:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970573:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x28:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970575:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970579:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x28:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970583:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x29:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970585:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970589:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x29:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970592:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x2a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970594:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970598:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x2a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970602:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x2b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970604:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970607:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x2b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970611:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x2c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970613:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970616:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x2c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970620:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x2d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970622:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970626:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x2d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970631:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x2e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970633:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970638:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x2e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970642:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x2f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970645:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970650:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x2f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970654:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x30:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970657:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970663:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x30:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970668:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x31:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970670:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970677:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x31:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970687:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x32:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970690:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970695:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x32:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970704:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x33:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970709:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970717:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x33:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970724:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x34:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970729:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970736:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x34:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970742:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x35:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970747:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970755:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x35:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970761:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x36:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970764:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970769:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x36:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970773:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x37:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970776:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970780:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x37:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970786:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x38:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970788:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970793:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x38:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970798:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x39:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970800:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970804:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x39:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970808:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x3a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970813:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970820:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x3a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970827:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x3b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970861:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970871:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x3b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970876:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x3c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970879:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970885:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x3c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970890:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x3d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970895:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970904:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x3d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970914:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x3e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970920:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970928:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x3e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970938:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x3f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970944:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970952:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x3f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970961:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x40:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970967:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.970977:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x40:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.970989:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x41:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.970995:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971008:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x41:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971021:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x42:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000008:00010000:0.0:1787827241.971025:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000004ed82aaa ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bad438eb/0xebdcd48a4298da8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d91ac expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00020000:00200000:2.0:1787827241.971027:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971039:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x42:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971050:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x43:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000008:00010000:0.0:1787827241.971053:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000004ed82aaa ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000bad438eb/0xebdcd48a4298da8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d91ac expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00020000:00200000:2.0:1787827241.971056:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971065:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x43:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971072:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x44:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971075:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971081:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x44:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971086:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x45:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:0.0:1787827241.971089:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476675840:192.168.203.140@tcp:4:cp.0 00020000:00200000:2.0:1787827241.971090:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.971098:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000d2558a36 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000fceb8461/0xebdcd48a4298dfbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x77:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9f2e expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.971107:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000d2558a36 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000fceb8461/0xebdcd48a4298dfbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x77:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9f2e expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.971120:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000067a96828 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000083301894/0xebdcd48a4298d9fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9024 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787827241.971129:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000fceb8461/0xebdcd48a4298dfbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x77:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9f2e expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.971163:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000067a96828 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000083301894/0xebdcd48a4298d9fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x62:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9024 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00000008:00100000:2.0:1787827241.971176:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce1680 - object 0x280000401:119 offset 0<>2129 00000100:00100000:0.0:1787827241.971183:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476675328:192.168.203.140@tcp:4:cp.0 00000100:00100000:0.0:1787827241.971194:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000008:00010000:2.0:1787827241.971195:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000004a8b27ee ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000044bcd5e7/0xebdcd48a4298de1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9aea expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.971206:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000004a8b27ee ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000044bcd5e7/0xebdcd48a4298de1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9aea expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.971207:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476677760:192.168.203.140@tcp:4:cp.0 00010000:00010000:2.0:1787827241.971244:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000044bcd5e7/0xebdcd48a4298de1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9aea expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.971281:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000003542bbd0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b342868b/0xebdcd48a4298daf6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d92bd expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.971281:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce0f00 - object 0x280000401:113 offset 0<>4117 00000080:00200000:2.0:1787827241.971306:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x45:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:0.0:1787827241.971307:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000003542bbd0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b342868b/0xebdcd48a4298daf6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d92bd expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.971317:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x46:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:1.0:1787827241.971322:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00020000:00200000:2.0:1787827241.971323:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.971333:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476676224:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.971335:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000db04fed2 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006800a193/0xebdcd48a4298dd26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d986d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787827241.971339:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476677888:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.971354:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000db04fed2 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006800a193/0xebdcd48a4298dd26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d986d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.971362:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000025cf9bfe ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000009ee0954d/0xebdcd48a4298e028 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da03f expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.971372:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000025cf9bfe ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000009ee0954d/0xebdcd48a4298e028 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da03f expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.971376:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006800a193/0xebdcd48a4298dd26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x6d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d986d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.971408:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce2d00 - object 0x2c0000401:109 offset 0<>2192 00010000:00010000:0.0:1787827241.971409:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000009ee0954d/0xebdcd48a4298e028 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da03f expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.971433:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x46:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:1.0:1787827241.971439:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678016:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.971445:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x47:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000008:00100000:0.0:1787827241.971453:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3e1c0 - object 0x2c0000401:120 offset 0<>423 00020000:00200000:2.0:1787827241.971453:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971465:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x47:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:1.0:1787827241.971479:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000080:00200000:2.0:1787827241.971479:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x48:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971488:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1787827241.971492:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678144:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.971511:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x48:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971520:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x49:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971525:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971532:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x49:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971537:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971541:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971546:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971550:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971553:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971558:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971565:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971569:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971576:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971583:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971588:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971596:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971605:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971610:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971619:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971625:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971630:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971637:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971642:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x51:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971644:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971648:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x51:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971655:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x52:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971657:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971661:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x52:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971666:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971670:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971674:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971678:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971682:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971687:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971691:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971694:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971701:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971710:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971714:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971723:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971730:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971735:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971745:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971754:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971759:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971767:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971774:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971778:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971783:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971790:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971792:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971797:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971802:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971805:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971814:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971823:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971858:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971871:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971880:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971884:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971889:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971897:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971902:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971911:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971922:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971927:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971936:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971944:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0xf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971949:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971959:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0xf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971969:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x10:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.971974:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.971985:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x10:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.971995:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x11:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972002:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:1.0:1787827241.972013:0:10936:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000a8520cc0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e87b1566/0xebdcd48a4298da6a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9151 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1787827241.972014:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x11:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972033:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x12:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000008:00010000:1.0:1787827241.972040:0:10936:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000a8520cc0 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000e87b1566/0xebdcd48a4298da6a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9151 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 1 00020000:00200000:2.0:1787827241.972041:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972054:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x12:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:0.0:1787827241.972064:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000d5a5292a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000abb2908a/0xebdcd48a4298db19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9318 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:1.0:1787827241.972064:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476675712:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.972065:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x13:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972077:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972089:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x13:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:0.0:1787827241.972094:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000d5a5292a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000abb2908a/0xebdcd48a4298db19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9318 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.972098:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x14:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972108:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972117:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x14:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972127:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:0.0:1787827241.972129:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476676352:192.168.203.140@tcp:4:cp.0 00020000:00200000:2.0:1787827241.972131:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.972141:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000009fb55a99 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003dae1a73/0xebdcd48a4298ddd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x70:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9a34 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.972151:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000009fb55a99 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003dae1a73/0xebdcd48a4298ddd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x70:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9a34 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1787827241.972158:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000004df7eb6d ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000514aa605/0xebdcd48a4298db5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d93ce expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:3.0:1787827241.972166:0:10935:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000000af7b30c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000eeaaa337/0xebdcd48a4298db3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9373 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787827241.972174:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003dae1a73/0xebdcd48a4298ddd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x70:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9a34 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.972176:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000001d8182b7 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000985be67/0xebdcd48a4298db82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9429 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.972181:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000004df7eb6d ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000514aa605/0xebdcd48a4298db5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d93ce expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:3.0:1787827241.972202:0:10935:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000000af7b30c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000eeaaa337/0xebdcd48a4298db3c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x66:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9373 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.972222:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000001d8182b7 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000985be67/0xebdcd48a4298db82 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x67:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9429 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.972224:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce30c0 - object 0x280000401:112 offset 0<>269 00000100:00100000:1.0:1787827241.972243:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476676608:192.168.203.140@tcp:4:cp.0 00000100:00100000:0.0:1787827241.972252:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f4f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476676736:192.168.203.140@tcp:4:cp.0 00000100:00100000:3.0:1787827241.972253:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476676480:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.972257:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000021372946 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a7c7b06e/0xebdcd48a4298e27b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da64a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.972266:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000008:00010000:2.0:1787827241.972277:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000021372946 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a7c7b06e/0xebdcd48a4298e27b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da64a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.972278:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476678272:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.972364:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000055ac6c3a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000041879d4a/0xebdcd48a4298df79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x76:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->36863) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e78 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.972380:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000055ac6c3a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000041879d4a/0xebdcd48a4298df79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x76:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->36863) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e78 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.972400:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a7c7b06e/0xebdcd48a4298e27b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da64a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:3.0:1787827241.972402:0:14467:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000f0faf625 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000533edf2/0xebdcd48a4298e29e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da6a5 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.972426:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000041879d4a/0xebdcd48a4298df79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x76:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->36863) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e78 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:3.0:1787827241.972466:0:14467:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000f0faf625 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000533edf2/0xebdcd48a4298e29e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da6a5 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00100000:0.0:1787827241.972512:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3f0c0 - object 0x280000401:118 offset 0<>35427 00010000:00010000:3.0:1787827241.972545:0:14467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000533edf2/0xebdcd48a4298e29e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da6a5 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.972555:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476678528:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.972570:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce21c0 - object 0x280000401:129 offset 0<>227516 00000100:00100000:1.0:1787827241.972606:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678400:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.972606:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972622:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972626:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.972635:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000d5b0dd8c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006a90b2ce/0xebdcd48a4298de3e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9b45 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.972645:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000d5b0dd8c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006a90b2ce/0xebdcd48a4298de3e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9b45 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00100000:3.0:1787827241.972647:0:14467:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a66f8780 - object 0x2c0000401:129 offset 0<>159678 00010000:00010000:2.0:1787827241.972674:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006a90b2ce/0xebdcd48a4298de3e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9b45 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:3.0:1787827241.972680:0:14467:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000078a97d1f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000075ff73a7/0xebdcd48a4298e2e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x82:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da75b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787827241.972681:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787827241.972692:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678656:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.972707:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce12c0 - object 0x2c0000401:113 offset 0<>3079 00000100:00100000:1.0:1787827241.972736:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476678784:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.972738:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972750:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972757:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972767:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972774:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972777:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972782:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972789:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972792:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972797:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972803:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972805:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972811:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972818:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x1b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972823:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972870:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x1b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:3.0:1787827241.972875:0:14467:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000078a97d1f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000075ff73a7/0xebdcd48a4298e2e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x82:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da75b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.972881:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x1c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972887:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972897:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x1c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972907:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x1d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972913:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972924:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x1d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:1.0:1787827241.972926:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000006ecad00a ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c1f046c4/0xebdcd48a4298dbc8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d94df expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.972934:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x1e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972940:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:1.0:1787827241.972944:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000006ecad00a ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c1f046c4/0xebdcd48a4298dbc8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d94df expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.972951:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x1e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:1.0:1787827241.972963:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476676992:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.972964:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x1f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.972975:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.972987:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x1f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.972997:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x20:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973002:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:0.0:1787827241.973003:0:10936:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000cfa2d7fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c87fa88a/0xebdcd48a4298dc0e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9595 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.973014:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000c0e3b4aa ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008c86884/0xebdcd48a4298ddb2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d99d9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.973028:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000c0e3b4aa ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008c86884/0xebdcd48a4298ddb2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d99d9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.973031:0:10936:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000cfa2d7fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000c87fa88a/0xebdcd48a4298dc0e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9595 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787827241.973053:0:14467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000075ff73a7/0xebdcd48a4298e2e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x82:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da75b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.973059:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476677248:192.168.203.140@tcp:4:cp.0 00010000:00010000:2.0:1787827241.973067:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008c86884/0xebdcd48a4298ddb2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x6f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d99d9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.973101:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce0780 - object 0x2c0000401:111 offset 0<>6213 00000008:00010000:0.0:1787827241.973104:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000009068126c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054c2b6dc/0xebdcd48a4298dc31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d95f0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.973118:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000089c8c79f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f952abaf/0xebdcd48a4298e3b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x85:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da97d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787827241.973124:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000008:00010000:0.0:1787827241.973126:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000009068126c ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000054c2b6dc/0xebdcd48a4298dc31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d95f0 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.973130:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000089c8c79f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f952abaf/0xebdcd48a4298e3b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x85:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da97d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787827241.973138:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476678912:192.168.203.140@tcp:4:cp.0 00010000:00010000:2.0:1787827241.973167:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f952abaf/0xebdcd48a4298e3b6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x85:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da97d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.973168:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476677376:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.973204:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce1a40 - object 0x2c0000401:133 offset 0<>2475 00000080:00200000:2.0:1787827241.973222:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x20:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:1.0:1787827241.973228:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476679168:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.973232:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x21:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973240:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973252:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x21:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973264:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x22:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000008:00100000:3.0:1787827241.973266:0:14467:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a66fa1c0 - object 0x2c0000401:130 offset 0<>255050 00020000:00200000:2.0:1787827241.973271:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973284:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x22:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973295:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x23:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:1.0:1787827241.973296:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00020000:00200000:2.0:1787827241.973301:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1787827241.973307:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476679040:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.973310:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x23:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973316:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x24:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973320:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973325:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x24:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973331:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x25:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973335:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973342:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x25:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973348:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x26:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973352:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973357:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x26:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973364:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x27:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973368:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973374:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x27:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973379:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x28:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973383:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973389:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x28:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973396:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x29:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973399:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973402:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x29:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973409:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973411:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973415:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973421:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973424:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973431:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973437:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973442:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973449:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973456:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973460:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973467:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973473:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973477:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973482:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973486:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x2f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973489:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973493:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x2f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973510:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x30:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973514:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973522:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x30:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973527:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x31:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973530:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973535:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x31:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973541:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x32:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973544:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973550:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x32:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973555:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x33:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973558:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973567:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x33:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973575:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x34:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973580:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973590:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x34:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973599:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x35:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973604:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.973615:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x35:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973623:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x36:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973629:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:1.0:1787827241.973632:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000043e2d6d1 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000008b9587d9/0xebdcd48a4298dbeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d953a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.973639:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x36:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:1.0:1787827241.973651:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000043e2d6d1 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000008b9587d9/0xebdcd48a4298dbeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x69:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d953a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.973653:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x37:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973665:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.973678:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000847e5969 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000001412e54d/0xebdcd48a4298e545 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db36a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787827241.973681:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476677120:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.973699:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000847e5969 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000001412e54d/0xebdcd48a4298e545 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db36a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1787827241.973713:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000054ea9985 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000b250d178/0xebdcd48a4298da47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->393215) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d90f6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787827241.973729:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000001412e54d/0xebdcd48a4298e545 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db36a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.973761:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce0000 - object 0x280000400:131 offset 0<>1735 00000008:00010000:1.0:1787827241.973781:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000054ea9985 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000b250d178/0xebdcd48a4298da47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x63:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->393215) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d90f6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.973790:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x37:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:1.0:1787827241.973796:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476675584:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.973802:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x38:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:0.0:1787827241.973807:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476679296:192.168.203.140@tcp:4:cp.0 00020000:00200000:2.0:1787827241.973809:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:1.0:1787827241.973817:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000025cf9bfe ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000009ee0954d/0xebdcd48a4298e028 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da03f expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.973824:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x38:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.973863:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x39:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.973869:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:2.0:1787827241.973880:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000000af43bb2 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000efac340c/0xebdcd48a4298e58b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x84:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db420 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.973890:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000000af43bb2 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000efac340c/0xebdcd48a4298e58b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x84:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db420 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827241.973939:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000efac340c/0xebdcd48a4298e58b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x84:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db420 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.973997:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce03c0 - object 0x280000400:132 offset 0<>38517 00000080:00200000:2.0:1787827241.974028:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x39:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.974041:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:0.0:1787827241.974047:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00020000:00200000:2.0:1787827241.974047:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.974060:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476679424:192.168.203.140@tcp:4:cp.0 00000008:00010000:1.0:1787827241.974090:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000025cf9bfe ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000009ee0954d/0xebdcd48a4298e028 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da03f expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.974093:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.974108:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974114:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1787827241.974117:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678144:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.974124:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.974132:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974136:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.974142:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:1.0:1787827241.974146:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000715e91d9 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d155b573/0xebdcd48a4298dc9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9701 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.974150:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974159:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.974167:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000008:00010000:1.0:1787827241.974176:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000715e91d9 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000d155b573/0xebdcd48a4298dc9a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9701 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787827241.974176:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974182:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.974188:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.974197:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x3f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000100:00100000:1.0:1787827241.974200:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476677632:192.168.203.140@tcp:4:cp.0 00020000:00200000:2.0:1787827241.974201:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.974210:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x3f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.974219:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x40:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974223:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:1.0:1787827241.974233:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000a4fc51fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000005bf881b/0xebdcd48a4298dc54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d964b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.974234:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000062d1880c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000232388ff/0xebdcd48a4298df56 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x75:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->114687) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e1d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.974264:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000062d1880c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000232388ff/0xebdcd48a4298df56 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x75:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->114687) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e1d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.974265:0:10935:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000004ed475b ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007a77bb05/0xebdcd48a4298dab0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9207 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.974265:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000a4fc51fb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000005bf881b/0xebdcd48a4298dc54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->16383) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d964b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:1.0:1787827241.974301:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476677504:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.974344:0:10935:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000004ed475b ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000007a77bb05/0xebdcd48a4298dab0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x64:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9207 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787827241.974370:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000232388ff/0xebdcd48a4298df56 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x75:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->114687) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e1d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.974379:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476675968:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.974458:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce3c00 - object 0x2c0000401:117 offset 0<>111139 00000008:00010000:2.0:1787827241.974488:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000d7cd9d32 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062074f98/0xebdcd48a4298e6c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db753 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.974511:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000d7cd9d32 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062074f98/0xebdcd48a4298e6c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db753 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.974520:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476679552:192.168.203.140@tcp:4:cp.0 00010000:00010000:2.0:1787827241.974534:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062074f98/0xebdcd48a4298e6c6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db753 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:2.0:1787827241.974566:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7886ce3480 - object 0x2c0000400:137 offset 0<>1754 00000008:00010000:2.0:1787827241.974583:0:38:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000a033ee24 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000070038913/0xebdcd48a4298e212 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da539 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.974585:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000008:00010000:2.0:1787827241.974596:0:38:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000a033ee24 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000070038913/0xebdcd48a4298e212 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da539 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.974600:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7886ce3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476679680:192.168.203.140@tcp:4:cp.0 00010000:00010000:2.0:1787827241.974712:0:38:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000070038913/0xebdcd48a4298e212 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->217087) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da539 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.974863:0:39964:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000037c3155e ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000066062073/0xebdcd48a4298e70c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db809 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00100000:2.0:1787827241.974868:0:38:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78841d8f00 - object 0x2c0000401:127 offset 0<>215293 00000008:00010000:1.0:1787827241.974875:0:39964:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000037c3155e ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000066062073/0xebdcd48a4298e70c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db809 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787827241.974896:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x40:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:1.0:1787827241.974900:0:39964:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000066062073/0xebdcd48a4298e70c lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db809 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.974910:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476679808:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.974910:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x41:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974922:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00100000:1.0:1787827241.974936:0:39964:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890b1e1c0 - object 0x2c0000400:138 offset 0<>1613 00000080:00200000:2.0:1787827241.974936:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x41:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.974951:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x42:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974956:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.974968:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x42:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000100:00100000:0.0:1787827241.974978:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000080:00200000:2.0:1787827241.974979:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x43:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.974987:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787827241.974991:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476679936:192.168.203.140@tcp:4:cp.0 00000080:00200000:2.0:1787827241.974999:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x43:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975010:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x44:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975016:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975027:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x44:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975038:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x45:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975044:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975054:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x45:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975062:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x46:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975065:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975071:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x46:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975075:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x47:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975078:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975082:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x47:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975086:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x48:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975089:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975093:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x48:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975098:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x49:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975100:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975104:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x49:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975109:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975112:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975116:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975121:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975124:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975129:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975135:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975138:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975145:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975150:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975154:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975159:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975163:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x4e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975166:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975170:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x4e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975174:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x50:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975176:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975180:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x50:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975186:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x240001b71:0x51:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975189:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975193:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x240001b71:0x51:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975299:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975304:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787827241.975313:0:38:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b72:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787827241.975318:0:38:0:(vvp_io.c:1889:vvp_io_init()) [0x200001b72:0x15:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787827241.975321:0:38:0:(lov_io.c:1253:lov_io_iter_init()) component[0] flags 0x10 00000008:00010000:0.0:1787827241.975955:0:10936:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000066c4d22f ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000f96f35de/0xebdcd48a4298dba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9484 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.975990:0:10936:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000066c4d22f ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000f96f35de/0xebdcd48a4298dba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x68:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->28671) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9484 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.976018:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476676864:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.976041:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000fcf6985d ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e775 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db91a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.976050:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000fcf6985d ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e775 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db91a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.976076:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000336f2fd7/0xebdcd48a4298e775 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x8b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db91a expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:0.0:1787827241.976115:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3ed00 - object 0x280000400:139 offset 0<>2858 00000008:00010000:0.0:1787827241.976164:0:10936:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000c0e3b4aa ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008c86884/0xebdcd48a4298ddb2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d99d9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.976181:0:10936:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000c0e3b4aa ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008c86884/0xebdcd48a4298ddb2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d99d9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.976200:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476678912:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.976219:0:10936:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000d5b0dd8c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006a90b2ce/0xebdcd48a4298de3e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9b45 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.976231:0:10936:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000d5b0dd8c ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006a90b2ce/0xebdcd48a4298de3e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9b45 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.976239:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000009fb55a99 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003dae1a73/0xebdcd48a4298ddd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x70:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9a34 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.976245:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476678784:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.976274:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000001fa576ea ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac599a08/0xebdcd48a4298e86a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbb97 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.976275:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000009fb55a99 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003dae1a73/0xebdcd48a4298ddd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x70:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9a34 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.976299:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000001fa576ea ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac599a08/0xebdcd48a4298e86a lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbb97 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787827241.976315:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476678272:192.168.203.140@tcp:4:cp.0 00010000:00010000:0.0:1787827241.976328:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ac599a08/0xebdcd48a4298e86a lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x8f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbb97 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.976341:0:33868:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000001b1431be ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000057cd9db4/0xebdcd48a4298dcbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->20479) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d975c expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:2.0:1787827241.976352:0:33868:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000001b1431be ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000057cd9db4/0xebdcd48a4298dcbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->20479) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d975c expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00100000:0.0:1787827241.976368:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3cf00 - object 0x2c0000400:143 offset 0<>6813 00010000:00010000:2.0:1787827241.976383:0:33868:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000057cd9db4/0xebdcd48a4298dcbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->20479) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d975c expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.976392:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476680064:192.168.203.140@tcp:4:cp.0 00000008:00100000:2.0:1787827241.976426:0:33868:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78841dad00 - object 0x280000401:108 offset 0<>18269 00000008:00010000:0.0:1787827241.976458:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000089c8c79f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f952abaf/0xebdcd48a4298e3b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x85:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da97d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:2.0:1787827241.976467:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000008:00010000:0.0:1787827241.976474:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000089c8c79f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000f952abaf/0xebdcd48a4298e3b6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x85:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da97d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:2.0:1787827241.976482:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476680320:192.168.203.140@tcp:4:cp.0 00000100:00100000:0.0:1787827241.976493:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476679168:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.976531:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000db04fed2 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006800a193/0xebdcd48a4298dd26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d986d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.976546:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000db04fed2 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000006800a193/0xebdcd48a4298dd26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x6d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d986d expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:0.0:1787827241.976556:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678016:192.168.203.140@tcp:4:cp.0 00000100:00100000:0.0:1787827241.976566:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787827241.976573:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476680192:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.976633:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000092291029 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b7f40aec/0xebdcd48a4298e8b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbc4d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.976646:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000092291029 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b7f40aec/0xebdcd48a4298e8b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbc4d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1787827241.976650:0:10935:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000693bd8f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c827a304/0xebdcd48a4298dad3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9262 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827241.976680:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b7f40aec/0xebdcd48a4298e8b0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbc4d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:0.0:1787827241.976708:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3d680 - object 0x2c0000400:144 offset 0<>269 00000008:00010000:0.0:1787827241.976730:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000f5e83eba ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a72a0ecc/0xebdcd48a4298e370 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x84:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da8c7 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.976741:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000f5e83eba ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a72a0ecc/0xebdcd48a4298e370 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x84:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da8c7 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787827241.976749:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476680448:192.168.203.140@tcp:4:cp.0 00010000:00010000:0.0:1787827241.977137:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000a72a0ecc/0xebdcd48a4298e370 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x84:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1204223) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da8c7 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:3.0:1787827241.977190:0:10935:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000693bd8f7 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000c827a304/0xebdcd48a4298dad3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x65:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9262 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:3.0:1787827241.977217:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a65f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476676096:192.168.203.140@tcp:4:cp.0 00000008:00010000:3.0:1787827241.977239:0:14467:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000079e566fe ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000806c7f21/0xebdcd48a4298e307 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->45055) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da7b6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1787827241.977250:0:14467:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000079e566fe ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000806c7f21/0xebdcd48a4298e307 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->45055) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da7b6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827241.977288:0:14467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000806c7f21/0xebdcd48a4298e307 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x83:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->45055) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da7b6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:3.0:1787827241.977346:0:14467:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a66fad00 - object 0x280000401:131 offset 0<>42517 00000100:00100000:1.0:1787827241.977386:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476680704:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.977640:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000d7cd9d32 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062074f98/0xebdcd48a4298e6c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db753 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.977655:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000d7cd9d32 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000062074f98/0xebdcd48a4298e6c6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x89:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db753 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:2.0:1787827241.977671:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476679680:192.168.203.140@tcp:4:cp.0 00000008:00100000:0.0:1787827241.977771:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3e940 - object 0x2c0000401:132 offset 0<>1200870 00000008:00010000:3.0:1787827241.977779:0:10936:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 000000004a8b27ee ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000044bcd5e7/0xebdcd48a4298de1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9aea expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:0.0:1787827241.977806:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000012ad067b ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008bcdff2/0xebdcd48a4298e8f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x91:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbd03 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787827241.977809:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476680576:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.977824:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000012ad067b ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008bcdff2/0xebdcd48a4298e8f6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x91:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbd03 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.977871:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000008bcdff2/0xebdcd48a4298e8f6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x91:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbd03 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:0.0:1787827241.977907:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3f840 - object 0x2c0000400:145 offset 0<>4117 00000008:00010000:1.0:1787827241.978016:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000055ac6c3a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000041879d4a/0xebdcd48a4298df79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x76:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->36863) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e78 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.978043:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000055ac6c3a ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000041879d4a/0xebdcd48a4298df79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x76:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->36863) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9e78 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:1.0:1787827241.978066:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476678528:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.978067:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000f0faf625 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000533edf2/0xebdcd48a4298e29e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da6a5 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.978090:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 00000000d2558a36 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000fceb8461/0xebdcd48a4298dfbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x77:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9f2e expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.978106:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000d2558a36 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000fceb8461/0xebdcd48a4298dfbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x77:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9f2e expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:1.0:1787827241.978117:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476677760:192.168.203.140@tcp:4:cp.0 00000100:00100000:1.0:1787827241.978123:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000008:00010000:0.0:1787827241.978131:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 00000000f0faf625 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000533edf2/0xebdcd48a4298e29e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->159743) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da6a5 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:1.0:1787827241.978134:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476680832:192.168.203.140@tcp:4:cp.0 00000100:00100000:0.0:1787827241.978163:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66f8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678656:192.168.203.140@tcp:4:cp.0 00000008:00010000:0.0:1787827241.978186:0:16762:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000e6f3e8eb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ba232b40/0xebdcd48a4298e3fc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x86:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daa33 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:0.0:1787827241.978198:0:16762:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000e6f3e8eb ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ba232b40/0xebdcd48a4298e3fc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x86:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daa33 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1787827241.978202:0:39964:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000001b1a7ff2 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003acf2d61/0xebdcd48a4298e005 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9fe4 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1787827241.978219:0:39964:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000001b1a7ff2 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003acf2d61/0xebdcd48a4298e005 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9fe4 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827241.978234:0:16762:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000ba232b40/0xebdcd48a4298e3fc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x86:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daa33 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827241.978275:0:39964:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000003acf2d61/0xebdcd48a4298e005 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x78:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->40959) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9fe4 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:0.0:1787827241.978276:0:16762:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890a3cb40 - object 0x2c0000401:134 offset 0<>1611 00000008:00100000:1.0:1787827241.978335:0:39964:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890b1ed00 - object 0x280000401:120 offset 0<>37842 00000008:00010000:1.0:1787827241.978359:0:39964:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000af3f9199 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089714b9a/0xebdcd48a4298e847 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->57343) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbb3c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1787827241.978363:0:10936:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 000000004a8b27ee ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000044bcd5e7/0xebdcd48a4298de1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x71:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d9aea expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.978373:0:39964:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000af3f9199 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089714b9a/0xebdcd48a4298e847 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->57343) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbb3c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.978377:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476681088:192.168.203.140@tcp:4:cp.0 00000100:00100000:3.0:1787827241.978389:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476677888:192.168.203.140@tcp:4:cp.0 00000100:00100000:3.0:1787827241.978399:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787827241.978410:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476680960:192.168.203.140@tcp:4:cp.0 00010000:00010000:1.0:1787827241.978430:0:39964:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 0000000089714b9a/0xebdcd48a4298e847 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x8e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->57343) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbb3c expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:1.0:1787827241.978516:0:39964:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890b1c780 - object 0x280000400:142 offset 0<>55540 00000008:00010000:1.0:1787827241.978534:0:39964:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000d4d70642 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000724b6f07/0xebdcd48a4298dd49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d98c8 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1787827241.978542:0:39964:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000d4d70642 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000724b6f07/0xebdcd48a4298dd49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x6e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d98c8 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.978550:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476681216:192.168.203.140@tcp:4:cp.0 00010000:00010000:1.0:1787827241.978569:0:39964:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000724b6f07/0xebdcd48a4298dd49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8d98c8 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:1.0:1787827241.978604:0:39964:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890b1d2c0 - object 0x280000401:110 offset 0<>938 00000008:00010000:3.0:1787827241.978702:0:10936:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000037c3155e ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000066062073/0xebdcd48a4298e70c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db809 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:3.0:1787827241.978723:0:10936:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000037c3155e ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000066062073/0xebdcd48a4298e70c lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x8a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8db809 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.978875:0:10937:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000092291029 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b7f40aec/0xebdcd48a4298e8b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbc4d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.978889:0:10937:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000092291029 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 00000000b7f40aec/0xebdcd48a4298e8b0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbc4d expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:1.0:1787827241.978903:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476680448:192.168.203.140@tcp:4:cp.0 00000100:00100000:1.0:1787827241.978909:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787827241.978915:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476681344:192.168.203.140@tcp:4:cp.0 00000100:00100000:3.0:1787827241.979027:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476679936:192.168.203.140@tcp:4:cp.0 00000008:00010000:3.0:1787827241.979056:0:14467:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1787827241.979072:0:10934:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9a78a6512000: committing for last_committed 4294967432 gen 1 00000100:00100000:1.0:1787827241.979133:0:10934:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9a7886ce1680 x1874671476677760/t4294967433(4294967433) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 2624/448 e 0 to 0 dl 1787827257 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00000008:00010000:3.0:1787827241.979138:0:14467:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000220af0ca ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1787827241.979173:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000021372946 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a7c7b06e/0xebdcd48a4298e27b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da64a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:1.0:1787827241.979243:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000021372946 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000a7c7b06e/0xebdcd48a4298e27b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x81:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->229375) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da64a expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:1.0:1787827241.979259:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7886ce21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476678400:192.168.203.140@tcp:4:cp.0 00000008:00010000:1.0:1787827241.979283:0:39964:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 000000002d35b46b ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000009f81abd8/0xebdcd48a4298e8d3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbca8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1787827241.979291:0:39964:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 000000002d35b46b ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000009f81abd8/0xebdcd48a4298e8d3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbca8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827241.979313:0:39964:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000009f81abd8/0xebdcd48a4298e8d3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbca8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:1.0:1787827241.979339:0:39964:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a7890b1cb40 - object 0x280000400:144 offset 0<>234 00000100:00100000:1.0:1787827241.979364:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476681472:192.168.203.140@tcp:4:cp.0 00000008:00010000:2.0:1787827241.979395:0:10934:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000078a97d1f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000075ff73a7/0xebdcd48a4298e2e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x82:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da75b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00010000:2.0:1787827241.979532:0:10934:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000078a97d1f ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 0000000075ff73a7/0xebdcd48a4298e2e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x82:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->258047) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da75b expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000100:00100000:2.0:1787827241.979551:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66fa1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476679040:192.168.203.140@tcp:4:cp.0 00010000:00010000:3.0:1787827241.979629:0:14467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000003d225f7b/0xebdcd48a4298e442 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x87:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->319487) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8daae9 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:3.0:1787827241.979793:0:14467:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a66f9680 - object 0x2c0000401:135 offset 0<>317548 00000008:00010000:3.0:1787827241.979817:0:14467:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 0000000059428622 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000674e675/0xebdcd48a4298e982 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbe6f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787827241.979851:0:10935:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787827241.979864:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476681600:192.168.203.140@tcp:4:cp.0 00000008:00010000:3.0:1787827241.979930:0:14467:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000059428622 ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000674e675/0xebdcd48a4298e982 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbe6f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827241.979965:0:14467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000674e675/0xebdcd48a4298e982 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x93:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->12287) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8dbe6f expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:3.0:1787827241.980093:0:14467:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a66fb840 - object 0x2c0000400:147 offset 0<>8441 00000008:00010000:3.0:1787827241.980116:0:14467:0:(osc_cache.c:2020:try_to_add_extent_for_io()) ### extent: 00000000f2fcd4bd ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000863a79e/0xebdcd48a4298e0fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da261 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1787827241.980128:0:14467:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 00000000f2fcd4bd ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000863a79e/0xebdcd48a4298e0fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x7b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da261 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787827241.980138:0:10934:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787827241.980149:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fb840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476681728:192.168.203.140@tcp:4:cp.0 00010000:00010000:3.0:1787827241.980156:0:14467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000863a79e/0xebdcd48a4298e0fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000020000 nid: local remote: 0x4059aa0aef8da261 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00000008:00100000:3.0:1787827241.980347:0:14467:0:(osc_request.c:2124:osc_brw_prep_request()) brw rpc ffff9a78a66f8b40 - object 0x2c0000401:123 offset 0<>1654 00000100:00100000:3.0:1787827241.980376:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66f8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476681856:192.168.203.140@tcp:4:cp.0 00010000:00010000:2.0:1787827242.185998:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827243.210045:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:3510e4c2-7f35-4dff-9360-35db1135010d:10937:1874671476681984:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827243.210066:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476682240:192.168.203.140@tcp:400:kworker.0 00000100:00100000:2.0:1787827243.210072:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476682112:192.168.203.140@tcp:400:kworker.0 00000100:00100000:1.0:1787827243.210209:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476682496:192.168.203.140@tcp:400:kworker.0 00010000:00010000:3.0:1787827243.210228:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827243.210251:0:10937:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787827243.210261:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476682368:192.168.203.140@tcp:400:kworker.0 00010000:00010000:2.0:1787827244.234024:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827245.258013:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827246.282008:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827247.306012:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827248.330041:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827248.330114:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3510e4c2-7f35-4dff-9360-35db1135010d:10934:1874671476682624:192.168.203.140@tcp:400:kworker.0 00000100:00100000:1.0:1787827248.330114:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476682880:192.168.203.140@tcp:400:kworker.0 00000100:00100000:2.0:1787827248.330151:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476683008:192.168.203.140@tcp:400:kworker.0 00000100:00100000:3.0:1787827248.330220:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476682752:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827248.330302:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476683136:192.168.203.140@tcp:400:kworker.0 00010000:00010000:3.0:1787827249.353927:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827250.377987:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827251.402025:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827252.426054:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827253.450001:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827253.450099:0:10934:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476683648:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827253.450103:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3510e4c2-7f35-4dff-9360-35db1135010d:10935:1874671476683264:192.168.203.140@tcp:400:kworker.0 00000100:00100000:3.0:1787827253.450103:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476683392:192.168.203.140@tcp:400:kworker.0 00000100:00100000:1.0:1787827253.450122:0:10937:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476683520:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827253.450316:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476683776:192.168.203.140@tcp:400:kworker.0 00010000:00010000:1.0:1787827254.474092:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827255.498071:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827256.522112:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:1.0:1787827257.545967:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890b1cb40 x1874671476681472/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 728/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00000400:2.0:1787827257.545967:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a78a66fb840 x1874671476681728/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 8936/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00000400:0.0:1787827257.545996:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a78a66f8b40 x1874671476681856/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 2144/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00000400:3.0:1787827257.546006:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a78a66f9680 x1874671476681600/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:0.0:1787827257.546063:0:10936:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787827257.546069:0:10936:0:(import.c:182:ptlrpc_set_import_discon()) lustre-OST0001-osc-ffff9a78a6512000: Connection to lustre-OST0001 (at 192.168.203.140@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787827257.546081:0:10936:0:(import.c:418:ptlrpc_pinger_force()) lustre-OST0001_UUID: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827257.546087:0:10935:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 1, was 1): DISCONN 00000100:00000400:3.0:1787827257.546100:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890b1c780 x1874671476681216/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00000400:0.0:1787827257.546105:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890a3cb40 x1874671476680960/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 2104/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:0.0:1787827257.546156:0:10936:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 1, was 1): DISCONN 00000100:00080000:3.0:1787827257.546161:0:10935:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from FULL to DISCONN 00000100:00000400:0.0:1787827257.546163:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a78841d8f00 x1874671476679808/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:02000400:3.0:1787827257.546172:0:10935:0:(import.c:182:ptlrpc_set_import_discon()) lustre-OST0000-osc-ffff9a78a6512000: Connection to lustre-OST0000 (at 192.168.203.140@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1787827257.546187:0:10935:0:(import.c:418:ptlrpc_pinger_force()) lustre-OST0000_UUID: waking up pinger s:DISCONN 00000100:00080000:0.0:1787827257.546192:0:10936:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 1, was 1): DISCONN 00000100:00000400:0.0:1787827257.546200:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7886ce03c0 x1874671476679424/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00000400:3.0:1787827257.546211:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890a3e940 x1874671476680576/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:0.0:1787827257.546213:0:10936:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 1, was 1): DISCONN 00000100:00000400:0.0:1787827257.546224:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7886ce0000 x1874671476679296/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 2224/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:0.0:1787827257.546265:0:10936:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 1, was 1): DISCONN 00000100:00080000:3.0:1787827257.546288:0:10935:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 1, was 1): DISCONN 00000100:00080000:0.0:1787827257.546370:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827257.546374:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827257.546377:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827257.546388:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1605 00000100:00100000:3.0:1787827257.546388:0:10935:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787827257.546422:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476684160:192.168.203.140@tcp:400:kworker.0 00000100:00080000:0.0:1787827257.546425:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827257.546445:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827257.546465:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476684288/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827257.546482:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827257.546486:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827257.546489:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827257.546495:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1605 00000100:00080000:0.0:1787827257.546504:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827257.546510:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827257.546532:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476684416/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787827257.546564:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827257.546600:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684288:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827257.546623:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:3510e4c2-7f35-4dff-9360-35db1135010d:10936:1874671476683904:192.168.203.140@tcp:400:kworker.0 00000100:00100000:3.0:1787827257.546651:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827257.546712:0:10936:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787827257.546722:0:10936:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476684032:192.168.203.140@tcp:400:kworker.0 00000100:00080000:1.0:1787827257.561719:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:1.0:1787827257.561734:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890b1d2c0 x1874671476681344/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 1432/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1787827257.561745:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:1.0:1787827257.561749:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890a3f840 x1874671476680832/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 4608/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1787827257.561757:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:1.0:1787827257.561762:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a78841dad00 x1874671476680320/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1787827257.561770:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00080000:2.0:1787827257.569873:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:2.0:1787827257.569885:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890b1ed00 x1874671476681088/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1787827257.569902:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:2.0:1787827257.569907:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a78a66fad00 x1874671476680704/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1787827257.569916:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:2.0:1787827257.569920:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890a3cf00 x1874671476680192/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 7304/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1787827257.569926:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:2.0:1787827257.569930:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7890a3ed00 x1874671476680064/t0(0) o4->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 3352/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1787827257.569941:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:2.0:1787827257.569945:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827241/real 1787827241] req@ffff9a7886ce3c00 x1874671476679552/t0(0) o4->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:6/4 lens 488/448 e 0 to 1 dl 1787827257 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1787827257.569952:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 2, was 1): CONNECTING 00010000:00010000:1.0:1787827258.570099:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:1.0:1787827259.593983:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827243/real 1787827243] req@ffff9a78841d8780 x1874671476682368/t0(0) o400->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787827259.593994:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:2.0:1787827259.594034:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827243/real 1787827243] req@ffff9a78841d92c0 x1874671476682240/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827259.594062:0:10935:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841d92c0 x1874671476682240/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827259.594069:0:10935:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1787827259.594073:0:10935:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0001-mdc-ffff9a78a6512000: Connection to lustre-MDT0001 (at 192.168.203.140@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00100000:1.0:1787827259.601624:0:10937:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841d8780 x1874671476682368/t0(0) o400->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827259.601633:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:1.0:1787827259.601639:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827243/real 1787827243] req@ffff9a78841d9a40 x1874671476682496/t0(0) o400->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827259.601647:0:10937:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841d9a40 x1874671476682496/t0(0) o400->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827259.601652:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:1.0:1787827259.601656:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827243/real 1787827243] req@ffff9a78841db0c0 x1874671476681984/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827259.601663:0:10937:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841db0c0 x1874671476681984/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827259.601668:0:10937:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from FULL to DISCONN 00000100:02020000:1.0:1787827259.601670:0:10937:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.140@tcp: Connection to MGS (at 192.168.203.140@tcp) was lost; in progress operations using this service will fail 00000100:00080000:2.0:1787827259.605349:0:10935:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0001_UUID: waking up pinger s:DISCONN 00000100:00100000:2.0:1787827259.605400:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476682240:192.168.203.140@tcp:400:kworker.0 00000100:00080000:1.0:1787827259.613057:0:10937:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.140@tcp for MGC192.168.203.140@tcp not replayable, auto-deactivating 00000100:00080000:1.0:1787827259.613060:0:10937:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:1.0:1787827259.613067:0:10937:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9a78841db0c0 x1874671476681984/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827259.613077:0:10937:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9a78841d8000 x1874671476682624/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827264 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827259.613088:0:10937:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9a7890b1e580 x1874671476683264/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827269 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827259.613101:0:10937:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9a7890a3e580 x1874671476683904/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1787827273 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827259.613116:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787827259.613120:0:10937:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:1.0:1787827259.613123:0:10937:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:1.0:1787827259.613126:0:10937:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1787827259.613130:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827259.613131:0:10937:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787827259.613131:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:3510e4c2-7f35-4dff-9360-35db1135010d:10934:1874671476682624:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827259.613133:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:3510e4c2-7f35-4dff-9360-35db1135010d:10936:1874671476683904:192.168.203.140@tcp:400:kworker.0 00000100:00080000:3.0:1787827259.613138:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827259.613154:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1605 00000100:00100000:1.0:1787827259.613162:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476682368:192.168.203.140@tcp:400:kworker.0 00000100:00100000:1.0:1787827259.613173:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476682496:192.168.203.140@tcp:400:kworker.0 00000100:00080000:3.0:1787827259.613173:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827259.613185:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:3510e4c2-7f35-4dff-9360-35db1135010d:10937:1874671476681984:192.168.203.140@tcp:400:kworker.0 00000100:00100000:3.0:1787827259.613214:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a66fa580 x1874671476684544/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827259.613233:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3510e4c2-7f35-4dff-9360-35db1135010d:10935:1874671476683264:192.168.203.140@tcp:400:kworker.0 00000100:00100000:2.0:1787827259.613268:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a66fa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476684544:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827259.613271:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.613277:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827259.613279:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827259.613285:0:10935:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476684672:192.168.203.140@tcp:400:kworker.0 00000100:00080000:3.0:1787827259.613287:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1605 00000100:00080000:3.0:1787827259.613299:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827259.613311:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827259.613328:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476684800/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827259.613340:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.613345:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827259.613349:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.613352:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827259.613364:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787827259.613366:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684800:192.168.203.140@tcp:38:kworker.0 00000100:00080000:3.0:1787827259.613369:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827259.613389:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.613393:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827259.613397:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.613403:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00000400:1.0:1787827259.634913:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827243/real 1787827243] req@ffff9a78841da1c0 x1874671476682112/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827259.634941:0:10934:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841da1c0 x1874671476682112/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827259 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827259.634949:0:10934:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787827259.634953:0:10934:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9a78a6512000: Connection to lustre-MDT0000 (at 192.168.203.140@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787827259.634959:0:10934:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0000_UUID: waking up pinger s:DISCONN 00000100:00100000:1.0:1787827259.634996:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476682112:192.168.203.140@tcp:400:kworker.0 00000100:00080000:3.0:1787827259.635015:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.635020:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827259.635023:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.635026:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827259.635029:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827259.635034:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1605 00000100:00080000:3.0:1787827259.635045:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827259.635055:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827259.635083:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476684928/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827259.635103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.635105:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787827259.635107:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827259.635109:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1787827259.635144:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684928:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787827260.617938:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827261.642086:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827262.665964:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827262.665973:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827262.665977:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827262.665979:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827262.665982:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787827262.665984:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827262.665985:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827262.665990:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827262.665993:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827262.665996:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827262.665998:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827263.689982:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827264.713934:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827264.713944:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827264.713947:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827264.713954:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827264.713957:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000400:2.0:1787827264.713957:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827248/real 1787827248] req@ffff9a78841db480 x1874671476683008/t0(0) o400->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000400:0.0:1787827264.713958:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827248/real 1787827248] req@ffff9a78841dbc00 x1874671476682880/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827264.713966:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827264.713971:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827264.713981:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827264.713987:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827264.713996:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827264.714021:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827264.714028:0:10936:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841dbc00 x1874671476682880/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000400:1.0:1787827264.714031:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827248/real 1787827248] req@ffff9a78841d83c0 x1874671476682752/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827264.714049:0:10936:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9a788849e800 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:1.0:1787827264.714065:0:10935:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841d83c0 x1874671476682752/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827264.714071:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476682880:192.168.203.140@tcp:400:kworker.0 00000100:00080000:1.0:1787827264.714079:0:10935:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9a788849e000 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:1.0:1787827264.714093:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476682752:192.168.203.140@tcp:400:kworker.0 00000100:00100000:2.0:1787827264.723760:0:10937:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841db480 x1874671476683008/t0(0) o400->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827264.723769:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:2.0:1787827264.723781:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476683008:192.168.203.140@tcp:400:kworker.0 00000100:00000400:1.0:1787827264.777947:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827248/real 1787827248] req@ffff9a78841d9e00 x1874671476683136/t0(0) o400->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827264.777986:0:10934:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78841d9e00 x1874671476683136/t0(0) o400->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827264 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827264.777994:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:1.0:1787827264.778015:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476683136:192.168.203.140@tcp:400:kworker.0 00010000:00010000:3.0:1787827265.738050:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827266.762053:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827267.786012:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827257/real 1787827257] req@ffff9a7890a3fc00 x1874671476684288/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827267 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787827267.786026:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827267.786062:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476684288/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827267 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827267.786075:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827257/real 1787827257] req@ffff9a7890a3c3c0 x1874671476684416/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827267 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827267.786091:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476684416/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827267 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827267.786115:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827267.786145:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827267.786150:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684288:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787827267.786158:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827267.786169:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787827267.786177:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827267.786178:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684416:192.168.203.140@tcp:8:kworker.0 00000100:00080000:2.0:1787827267.786186:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827267.786194:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827267.786197:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827267.786201:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827267.786203:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827267.786205:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827267.786208:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827267.786211:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827267.786220:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1630 00000100:00080000:2.0:1787827267.786229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787827267.786234:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827267.786244:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827267.786276:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476685056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827267.786300:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827267.786303:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827267.786304:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827267.786307:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1630 00000100:00080000:2.0:1787827267.786310:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787827267.786312:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827267.786315:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827267.786337:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476685184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827267.786388:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827267.786520:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827268.809946:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:2.0:1787827269.834000:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827253/real 1787827253] req@ffff9a7890b1f840 x1874671476683776/t0(0) o400->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787827269.834016:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:1.0:1787827269.834050:0:10937:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827253/real 1787827253] req@ffff9a7890b1c000 x1874671476683520/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827269.834081:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827259/real 1787827259] req@ffff9a78a6774b40 x1874671476684928/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827269.834087:0:10937:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a7890b1c000 x1874671476683520/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827269.834146:0:10937:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9a788849e800 already not connected (conn 2, was 1): CONNECTING 00000100:00000400:3.0:1787827269.834158:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827253/real 1787827253] req@ffff9a7890b1f480 x1874671476683392/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827269.834166:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476684928/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827269.834170:0:10937:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:97196825-2cb9-49ba-a47d-1455a997ce3a:10937:1874671476683520:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827269.834220:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827259/real 1787827259] req@ffff9a78a67752c0 x1874671476684800/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827269.834230:0:10936:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a7890b1f480 x1874671476683392/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827269.834241:0:10936:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9a788849e000 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:3.0:1787827269.834261:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476683392:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827269.834359:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476684800/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827269.834367:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827259/real 1787827259] req@ffff9a78a66fa580 x1874671476684544/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827269.834385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a66fa580 x1874671476684544/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827269.834403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827269.834441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827269.834458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684928:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787827269.834460:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827269.834467:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827269.834469:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827269.834471:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827269.834473:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827269.834478:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1632 00000100:00080000:1.0:1787827269.834487:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787827269.834492:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827269.834500:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1787827269.834506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787827269.834524:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476685312/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827269.834541:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827269.834544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787827269.834545:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827269.834549:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827269.834552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476684800:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787827269.834556:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1632 00000100:00080000:1.0:1787827269.834569:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787827269.834572:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827269.834577:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1787827269.834579:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787827269.834588:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476685440/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827269.834596:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827269.834600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787827269.834601:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827269.834606:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827269.834609:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827269.834617:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827269.834618:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a66fa580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476684544:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827269.834625:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827269.834628:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827269.834635:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1632 00000100:00080000:1.0:1787827269.834637:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787827269.834640:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827269.834647:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476685568/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827269.834653:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827269.834655:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827269.834657:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827269.834659:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787827269.834687:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685312:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827269.834815:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827269.834947:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476685568:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827269.844375:0:10935:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a7890b1f840 x1874671476683776/t0(0) o400->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827269.844385:0:10935:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f000 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:2.0:1787827269.844402:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476683776:192.168.203.140@tcp:400:kworker.0 00000100:00000400:1.0:1787827269.897963:0:10934:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827253/real 1787827253] req@ffff9a7890b1da40 x1874671476683648/t0(0) o400->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827269.898000:0:10934:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a7890b1da40 x1874671476683648/t0(0) o400->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 1 dl 1787827269 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827269.898006:0:10934:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9a788923f800 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:1.0:1787827269.898022:0:10934:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:97196825-2cb9-49ba-a47d-1455a997ce3a:10934:1874671476683648:192.168.203.140@tcp:400:kworker.0 00010000:00010000:3.0:1787827270.858042:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827271.882000:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827272.841979:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.841989:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827272.841992:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.841995:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827272.841998:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.842001:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827272.842003:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.842013:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827272.842016:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827272.842018:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827272.905931:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827273.929917:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1787827273.929921:0:10936:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827257/real 1787827257] req@ffff9a7890a3de00 x1874671476684032/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827273 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000400:2.0:1787827273.929930:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827257/real 1787827257] req@ffff9a7890a3f480 x1874671476684160/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827273 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827273.929975:0:10935:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a7890a3f480 x1874671476684160/t0(0) o400->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827273 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827273.929987:0:10935:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9a788849e800 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:2.0:1787827273.930013:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476684160:192.168.203.140@tcp:400:kworker.0 00000100:00100000:0.0:1787827273.940156:0:10936:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a7890a3de00 x1874671476684032/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827273 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827273.940166:0:10936:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9a788849e000 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:0.0:1787827273.940180:0:10936:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:97196825-2cb9-49ba-a47d-1455a997ce3a:10936:1874671476684032:192.168.203.140@tcp:400:kworker.0 00000100:00080000:1.0:1787827274.889935:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827274.889945:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827274.889948:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827274.889951:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827274.889953:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827274.889955:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827274.889959:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827274.889962:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827274.889964:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827274.889966:0:11718:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827274.953972:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:2.0:1787827275.977893:0:10935:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827259/real 1787827259] req@ffff9a78a6777480 x1874671476684672/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827275 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787827275.977904:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827275.977927:0:10935:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9a78a6777480 x1874671476684672/t0(0) o400->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 1 dl 1787827275 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827275.977935:0:10935:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9a788849e000 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:2.0:1787827275.977953:0:10935:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:97196825-2cb9-49ba-a47d-1455a997ce3a:10935:1874671476684672:192.168.203.140@tcp:400:kworker.0 00010000:00010000:3.0:1787827277.001901:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827278.025912:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827279.049913:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827280.073949:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827280.073959:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827280.073962:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827280.073965:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827280.073968:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827280.073971:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827280.073974:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827280.073977:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827280.073980:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827280.073984:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827280.073987:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827281.097920:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827282.057934:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827267/real 1787827267] req@ffff9a78841db480 x1874671476685056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827282 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827282.057979:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476685056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827282 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827282.057990:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827267/real 1787827267] req@ffff9a78841d8000 x1874671476685184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827282 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827282.058006:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476685184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827282 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827282.058028:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827282.058049:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827282.058052:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685056:192.168.203.140@tcp:8:kworker.0 00000100:00080000:0.0:1787827282.058059:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827282.058063:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827282.058066:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685184:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787827282.058087:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827282.058094:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827282.058097:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827282.058100:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827282.058103:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827282.058106:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827282.058108:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827282.058110:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827282.058113:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827282.058119:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1640 00000100:00080000:1.0:1787827282.058130:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 10s 00000100:00080000:1.0:1787827282.058136:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827282.058148:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827282.058178:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476685696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827282.058204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827282.058207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827282.058209:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827282.058215:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1640 00000100:00080000:1.0:1787827282.058231:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 10s 00000100:00080000:1.0:1787827282.058237:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827282.058243:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685696:192.168.203.140@tcp:8:kworker.0 00000008:00100000:1.0:1787827282.058244:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827282.058262:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476685824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827282.058359:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827282.121925:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827283.145943:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827284.106040:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827269/real 1787827269] req@ffff9a7890b1c000 x1874671476685312/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827284 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827284.106083:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476685312/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827284 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827284.106096:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827269/real 1787827269] req@ffff9a7890b1fc00 x1874671476685440/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827284 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827284.106109:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476685440/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827284 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827284.106119:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827269/real 1787827269] req@ffff9a7890b1c3c0 x1874671476685568/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827284 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827284.106137:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476685568/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827284 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827284.106156:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827284.106178:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827284.106183:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685312:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827284.106196:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827284.106203:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827284.106220:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685440:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827284.106226:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827284.106232:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827284.106237:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476685568:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827284.106238:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827284.106245:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827284.106248:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827284.106254:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1642 00000100:00080000:2.0:1787827284.106263:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787827284.106269:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827284.106302:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476685952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827284.106321:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827284.106323:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827284.106324:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827284.106327:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1642 00000100:00080000:2.0:1787827284.106330:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787827284.106332:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827284.106340:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827284.106348:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476686080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827284.106357:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476685952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827284.106358:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827284.106366:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827284.106369:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827284.106373:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1642 00000100:00080000:2.0:1787827284.106376:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787827284.106378:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827284.106382:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827284.106391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476686208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827284.106398:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827284.106401:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827284.106403:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827284.106406:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787827284.106477:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827284.106510:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686208:192.168.203.140@tcp:38:kworker.0 00010000:00010000:3.0:1787827284.169907:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827285.193926:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827286.217921:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827287.113956:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827287.113969:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827287.113974:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827287.113978:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827287.113982:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827287.113986:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827287.113989:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827287.113993:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827287.113997:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827287.114000:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827287.242026:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827288.265979:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827289.161968:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827289.162008:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827289.162014:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827289.162019:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827289.162023:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827289.162027:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827289.162031:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827289.162034:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787827289.162037:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827289.162041:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827289.289920:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827290.313904:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827291.337948:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827292.361933:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827293.385994:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827294.409936:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827294.409946:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827294.409949:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827294.409954:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827294.409958:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827294.409962:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827294.409963:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827294.409964:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827294.409969:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827294.409971:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827294.409973:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827295.433964:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827296.457904:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827297.481917:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827298.505906:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827299.529963:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827299.529974:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827299.529979:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827299.529983:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827299.529986:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827299.529989:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827299.529991:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827299.529994:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827299.529997:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827299.530000:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787827299.530036:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827300.553919:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:3.1F:1787827301.193932:0:10926:0:(socklnd_cb.c:2294:ksocknal_find_timed_out_conn()) Timeout sending data to 12345-192.168.203.140@tcp (192.168.203.140:988) the network or that node may be down. 00010000:00010000:3.0:1787827301.577918:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827302.473931:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827282/real 1787827282] req@ffff9a7890b1f840 x1874671476685824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827302 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827302.473964:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476685824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827302 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827302.473974:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827282/real 1787827282] req@ffff9a7890b1da40 x1874671476685696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827302 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827302.473989:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476685696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827302 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827302.474011:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827302.474029:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827302.474033:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685824:192.168.203.140@tcp:8:kworker.0 00000100:00080000:0.0:1787827302.474042:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827302.474049:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827302.474053:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476685696:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787827302.474130:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827302.474135:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827302.474138:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827302.474141:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827302.474144:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827302.474146:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827302.474147:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827302.474150:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827302.474152:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827302.474158:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1654 00000100:00080000:1.0:1787827302.474168:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787827302.474174:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827302.474185:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827302.474213:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476686336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827302.474232:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827302.474235:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827302.474236:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827302.474239:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1654 00000100:00080000:1.0:1787827302.474242:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787827302.474244:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827302.474247:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827302.474255:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476686464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827302.474272:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827302.474433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827302.601909:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827303.625903:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827304.521947:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827284/real 1787827284] req@ffff9a78841da580 x1874671476686080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827304 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827304.521975:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476686080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827304 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827304.521983:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827284/real 1787827284] req@ffff9a78841d8b40 x1874671476686208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827304 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827304.521991:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476686208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827304 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827304.521997:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787827284/real 1787827284] req@ffff9a78841d92c0 x1874671476685952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827304 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827304.522010:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476685952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827304 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827304.522026:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827304.522046:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827304.522050:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827304.522053:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827304.522060:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827304.522076:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686208:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827304.522081:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827304.522093:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827304.522121:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787827304.522121:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827304.522131:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827304.522138:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476685952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827304.522146:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1656 00000100:00080000:1.0:1787827304.522160:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787827304.522185:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827304.522234:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476686592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827304.522267:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827304.522272:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827304.522274:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827304.522278:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1656 00000100:00080000:1.0:1787827304.522283:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787827304.522288:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827304.522306:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827304.522322:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476686592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827304.522327:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476686720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827304.522343:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827304.522347:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827304.522350:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827304.522356:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1656 00000100:00080000:1.0:1787827304.522363:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787827304.522370:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827304.522379:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827304.522393:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476686848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827304.522407:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827304.522412:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827304.522416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827304.522421:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827304.522437:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827304.522441:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787827304.522444:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827304.522447:0:36:0:(pinger.c:247:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787827304.522473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827304.522508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686848:192.168.203.140@tcp:38:kworker.0 00010000:00010000:3.0:1787827304.649913:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827305.546235:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1020 -> 192.168.203.140:988 00000400:00000100:3.0:1787827305.546291:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827305.546304:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827305.546339:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827305.546345:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827305.546354:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827305.546367:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827305.546389:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827304/real 1787827305] req@ffff9a7890b1e940 x1874671476686720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827305.546415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476686720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827305.546427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827305.546434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827305.546438:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827305.546456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827304/real 1787827305] req@ffff9a7890b1f0c0 x1874671476686848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827305.546468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476686848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827305.546475:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827305.546478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827305.546482:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827305.546488:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827304/real 1787827305] req@ffff9a7890b1cf00 x1874671476686592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827305.546499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476686592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827305.546506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827305.546510:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827305.546513:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476686592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827305.546519:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827302/real 1787827305] req@ffff9a7890b1e580 x1874671476686464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827305.546530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476686464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827305.546540:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827305.546544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827305.546547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686464:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827305.546554:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827302/real 1787827305] req@ffff9a7890b1f480 x1874671476686336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827305.546572:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476686336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827305.546581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827305.546584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827305.546587:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476686336:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827305.673924:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827306.698019:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827307.529927:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827307.529941:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827307.529946:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827307.529958:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1677 00000100:00080000:1.0:1787827307.529969:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827307.529977:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827307.530019:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476686976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827307.530055:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827307.530058:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827307.530060:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827307.530065:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1677 00000100:00080000:1.0:1787827307.530069:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827307.530074:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827307.530085:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827307.530098:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476687104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827307.530102:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476686976:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827307.530119:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827307.530123:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827307.530124:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827307.530127:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1677 00000100:00080000:1.0:1787827307.530131:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827307.530134:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827307.530138:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827307.530147:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476687232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827307.530154:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827307.530158:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827307.530160:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827307.530167:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1675 00000100:00080000:1.0:1787827307.530183:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827307.530190:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827307.530196:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827307.530211:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476687360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827307.530222:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827307.530224:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827307.530226:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827307.530229:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1675 00000100:00080000:1.0:1787827307.530233:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827307.530239:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827307.530242:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687104:192.168.203.140@tcp:38:kworker.0 00000008:00100000:1.0:1787827307.530247:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827307.530264:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476687488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827307.530285:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687232:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827307.530316:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687360:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827307.530338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687488:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827307.721963:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827308.618205:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1020 -> 192.168.203.140:988 00000400:00000100:1.0:1787827308.618250:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827308.618268:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827308.618305:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827308.618313:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827308.618320:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827308.618326:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827308.618398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827307/real 1787827308] req@ffff9a7890b1cf00 x1874671476687488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827308.618429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476687488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827308.618445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827308.618451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827308.618457:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687488:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827308.618467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827307/real 1787827308] req@ffff9a7890b1de00 x1874671476687104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827308.618480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476687104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827308.618487:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827308.618491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827308.618494:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687104:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827308.618500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827307/real 1787827308] req@ffff9a7890b1f480 x1874671476687232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827308.618511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476687232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827308.618519:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827308.618522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827308.618525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687232:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827308.618531:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827307/real 1787827308] req@ffff9a7890b1e580 x1874671476687360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827308.618558:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476687360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827308.618569:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827308.618572:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827308.618576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687360:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827308.618583:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827307/real 1787827308] req@ffff9a7890b1d680 x1874671476686976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827308.618597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476686976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827332 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827308.618605:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827308.618609:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827308.618611:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476686976:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787827308.745993:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827309.769921:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827310.793963:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827311.817978:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827312.841950:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827312.841964:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827312.841970:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827312.841983:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1680 00010000:00010000:3.0:1787827312.841989:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827312.841998:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827312.842025:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827312.842101:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476687616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827312.842134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827312.842139:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827312.842142:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827312.842146:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1680 00000100:00080000:1.0:1787827312.842151:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827312.842157:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827312.842169:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827312.842179:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476687616:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827312.842190:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476687744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827312.842204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827312.842209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827312.842211:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827312.842215:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1680 00000100:00080000:1.0:1787827312.842220:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827312.842225:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827312.842231:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827312.842245:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476687872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827312.842258:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827312.842262:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827312.842264:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827312.842269:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1680 00000100:00080000:1.0:1787827312.842273:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827312.842279:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827312.842286:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827312.842303:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476688000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827312.842315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827312.842319:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827312.842322:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827312.842328:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687744:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787827312.842330:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1680 00000100:00080000:1.0:1787827312.842339:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827312.842346:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827312.842352:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827312.842369:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476688128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827312.842372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687872:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827312.842404:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688000:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827312.842450:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688128:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827313.866034:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827314.762186:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1020 -> 192.168.203.140:988 00000400:00000100:3.0:1787827314.762250:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827314.762271:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827314.762314:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827314.762326:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827314.762337:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827314.762348:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827314.762376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827312/real 1787827314] req@ffff9a7890b1c3c0 x1874671476688128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827314.762416:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476688128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827314.762437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827314.762444:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827314.762449:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688128:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827314.762460:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827312/real 1787827314] req@ffff9a7890b1e940 x1874671476687744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827314.762475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476687744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827314.762484:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827314.762488:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827314.762492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687744:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827314.762501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827312/real 1787827314] req@ffff9a7890b1da40 x1874671476687872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827314.762522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476687872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827314.762544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827314.762549:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827314.762554:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476687872:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827314.762562:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827312/real 1787827314] req@ffff9a7890b1f840 x1874671476688000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827314.762577:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476688000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827314.762588:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827314.762592:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827314.762596:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688000:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827314.762605:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827312/real 1787827314] req@ffff9a7890b1f0c0 x1874671476687616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827314.762619:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476687616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827314.762627:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827314.762631:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827314.762635:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476687616:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787827314.889934:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827315.913966:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827316.937926:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827317.961932:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827317.961952:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827317.961964:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827317.961982:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827317.961994:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1685 00000100:00080000:1.0:1787827317.962008:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827317.962017:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827317.962061:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476688256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827317.962092:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827317.962098:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827317.962101:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827317.962104:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1685 00000100:00080000:1.0:1787827317.962108:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827317.962112:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827317.962124:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827317.962138:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476688384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827317.962149:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827317.962152:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827317.962154:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827317.962160:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476688256:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827317.962162:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1685 00000100:00080000:1.0:1787827317.962170:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827317.962176:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827317.962181:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827317.962195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476688512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827317.962205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827317.962209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827317.962211:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827317.962215:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1685 00000100:00080000:1.0:1787827317.962234:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827317.962238:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827317.962244:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827317.962250:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827317.962258:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476688640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827317.962268:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827317.962272:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827317.962273:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827317.962278:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1685 00000100:00080000:1.0:1787827317.962281:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827317.962284:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827317.962289:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827317.962301:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476688768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827317.962314:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827317/real 1787827317] req@ffff9a7890b1fc00 x1874671476688256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827317.962341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476688256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827317.962353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827317.962368:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827317.962376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476688256:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827317.962401:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688384:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827317.962421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827317.962433:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827317/real 1787827317] req@ffff9a7890b1c000 x1874671476688384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827317.962447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476688384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827317.962456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827317.962459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827317.962463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688384:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827317.962470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688512:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827317.962490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827317.962500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827317/real 1787827317] req@ffff9a7890b1f0c0 x1874671476688512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827317.962508:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476688512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827317.962514:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827317.962516:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827317.962518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688512:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827317.962522:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688640:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827317.962535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827317.962541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827317/real 1787827317] req@ffff9a7890b1f840 x1874671476688640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827317.962548:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476688640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827317.962555:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827317.962557:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827317.962560:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688640:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827317.962563:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688768:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827317.962573:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827317.962578:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827317/real 1787827317] req@ffff9a7890b1da40 x1874671476688768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827317.962585:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476688768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827317.962592:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827317.962594:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827317.962596:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476688768:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827318.985900:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827320.009926:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827321.033920:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827322.057926:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827323.081972:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827323.081979:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827323.081991:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827323.081996:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827323.082005:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1690 00000100:00080000:2.0:1787827323.082015:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827323.082022:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827323.082065:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476688896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827323.082095:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827323.082099:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827323.082101:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827323.082105:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1690 00000100:00080000:2.0:1787827323.082109:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827323.082115:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827323.082126:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827323.082148:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476688896:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827323.082148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476689024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827323.082167:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827323.082171:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827323.082174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827323.082178:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1690 00000100:00080000:2.0:1787827323.082183:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827323.082188:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827323.082194:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827323.082206:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476689152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827323.082213:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827323.082216:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827323.082217:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827323.082221:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1690 00000100:00080000:2.0:1787827323.082224:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827323.082226:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827323.082231:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827323.082243:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476689280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827323.082251:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827323.082254:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827323.082256:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827323.082259:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1690 00000100:00080000:2.0:1787827323.082261:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827323.082263:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827323.082266:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827323.082268:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689024:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827323.082277:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476689408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827323.082304:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689152:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827323.082323:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689280:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827323.082345:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689408:192.168.203.140@tcp:8:kworker.0 00000400:00000100:3.0:1787827323.978346:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1020 -> 192.168.203.140:988 00000400:00000100:3.0:1787827323.978402:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827323.978416:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827323.978450:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827323.978462:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827323.978478:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827323.978489:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827323.978497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827323/real 1787827323] req@ffff9a78841da940 x1874671476689024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827323.978532:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476689024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827323.978551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827323.978564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827323.978569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689024:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827323.978578:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827323/real 1787827323] req@ffff9a78841d9680 x1874671476689152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827323.978591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476689152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827323.978600:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827323.978604:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827323.978608:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689152:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827323.978615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827323/real 1787827323] req@ffff9a78841d92c0 x1874671476689280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827323.978628:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476689280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827323.978639:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827323.978642:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827323.978645:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689280:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827323.978652:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827323/real 1787827323] req@ffff9a78841db840 x1874671476688896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827323.978664:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476688896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827323.978672:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827323.978676:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827323.978680:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476688896:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827323.978711:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827323/real 1787827323] req@ffff9a78841d8b40 x1874671476689408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827323.978724:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476689408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827323.978733:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827323.978737:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827323.978740:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689408:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827324.106023:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827325.129956:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827326.153990:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827327.177970:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827328.202035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827328.202044:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827328.202049:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827328.202059:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1695 00010000:00010000:3.0:1787827328.202070:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827328.202071:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827328.202082:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827328.202123:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476689536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827328.202153:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827328.202156:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827328.202157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827328.202160:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1695 00000100:00080000:1.0:1787827328.202163:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827328.202166:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827328.202177:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827328.202185:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476689664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827328.202193:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827328.202196:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827328.202198:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827328.202201:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1695 00000100:00080000:1.0:1787827328.202204:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827328.202206:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827328.202209:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827328.202218:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476689792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827328.202235:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476689536:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827328.202254:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827328.202258:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827328.202259:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827328.202263:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1695 00000100:00080000:1.0:1787827328.202266:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827328.202269:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827328.202274:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827328.202285:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476689920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827328.202299:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827328.202302:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827328.202304:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827328.202307:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1695 00000100:00080000:1.0:1787827328.202312:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827328.202315:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827328.202319:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:0.0:1787827328.202326:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827328.202331:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476690048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827328.202402:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827328/real 1787827328] req@ffff9a7890b1e940 x1874671476689536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827328.202419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476689536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827328.202429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827328.202433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827328.202436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476689536:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827328.202442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689664:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827328.202461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827328.202470:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827328/real 1787827328] req@ffff9a7890b1c3c0 x1874671476689664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827328.202479:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476689664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827328.202484:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827328.202486:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827328.202489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689664:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827328.202511:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689792:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827328.202523:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827328.202530:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827328/real 1787827328] req@ffff9a7890b1d680 x1874671476689792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827328.202538:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476689792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827328.202544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827328.202546:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827328.202548:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689792:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827328.202551:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689920:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827328.202563:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827328.202569:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827328/real 1787827328] req@ffff9a7890b1e580 x1874671476689920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827328.202582:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476689920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827328.202590:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827328.202592:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827328.202594:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476689920:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827328.202597:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690048:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827328.202608:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827328.202614:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827328/real 1787827328] req@ffff9a7890b1f480 x1874671476690048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827328.202622:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476690048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827328.202629:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827328.202631:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827328.202633:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690048:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827329.225913:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827330.249925:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827331.273915:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827332.298043:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827333.321985:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827333.321998:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827333.322004:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827333.322019:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1700 00010000:00010000:3.0:1787827333.322019:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827333.322038:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827333.322047:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827333.322088:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476690176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827333.322114:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827333.322117:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827333.322119:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827333.322122:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1700 00000100:00080000:1.0:1787827333.322124:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827333.322127:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827333.322138:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827333.322148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476690304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827333.322157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827333.322160:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827333.322161:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827333.322165:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1700 00000100:00080000:1.0:1787827333.322171:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827333.322172:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476690176:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827333.322176:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827333.322187:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827333.322198:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476690432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827333.322206:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827333.322209:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827333.322210:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827333.322215:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1700 00000100:00080000:1.0:1787827333.322234:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827333.322241:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827333.322248:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827333.322267:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476690560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827333.322274:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827333.322277:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827333.322279:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827333.322285:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1700 00000100:00080000:1.0:1787827333.322288:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827333.322293:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827333.322300:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827333.322304:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690304:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827333.322316:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476690688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827333.322336:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690432:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827333.322359:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690560:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827333.322377:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690688:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827334.346032:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827335.370020:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827336.266221:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1020 -> 192.168.203.140:988 00000400:00000100:3.0:1787827336.266278:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827336.266300:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827336.266350:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827336.266362:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827336.266372:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827336.266390:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827336.266420:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827333/real 1787827336] req@ffff9a7890b1d680 x1874671476690688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827336.266454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476690688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827336.266472:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827336.266479:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827336.266484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690688:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827336.266494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827333/real 1787827336] req@ffff9a7890b1cf00 x1874671476690304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827336.266507:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476690304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827336.266516:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827336.266521:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827336.266525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690304:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827336.266534:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827333/real 1787827336] req@ffff9a7890b1f480 x1874671476690432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827336.266547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476690432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827336.266554:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827336.266558:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827336.266562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690432:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827336.266571:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827333/real 1787827336] req@ffff9a7890b1e580 x1874671476690560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827336.266585:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476690560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827336.266595:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827336.266599:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827336.266602:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690560:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827336.266610:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827333/real 1787827336] req@ffff9a7890b1de00 x1874671476690176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827336.266624:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476690176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827336.266632:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827336.266636:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827336.266639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476690176:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787827336.393980:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827337.417971:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827338.441969:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787827338.441981:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827338.441982:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827338.441991:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827338.442006:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1705 00000100:00080000:2.0:1787827338.442019:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827338.442029:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827338.442080:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476690816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827338.442114:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827338.442119:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827338.442121:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827338.442125:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1705 00000100:00080000:2.0:1787827338.442130:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827338.442135:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827338.442148:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827338.442166:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476690816:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827338.442167:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476690944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827338.442185:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827338.442189:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827338.442192:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827338.442198:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1705 00000100:00080000:2.0:1787827338.442202:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827338.442207:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827338.442213:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827338.442225:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476691072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827338.442232:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827338.442235:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827338.442237:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827338.442240:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1705 00000100:00080000:2.0:1787827338.442243:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787827338.442248:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787827338.442249:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827338.442256:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827338.442265:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476691200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827338.442272:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827338.442275:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827338.442277:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827338.442279:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1705 00000100:00080000:2.0:1787827338.442282:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827338.442285:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827338.442288:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827338.442295:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476691328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827338.442320:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827338/real 1787827338] req@ffff9a78841da580 x1874671476690816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827338.442337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476690816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827338.442346:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827338.442350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827338.442354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476690816:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827338.442376:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690944:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827338.442395:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827338.442405:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827338/real 1787827338] req@ffff9a78841d8000 x1874671476690944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827338.442414:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476690944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827338.442420:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827338.442422:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827338.442425:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476690944:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827338.442428:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691072:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827338.442440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827338.442446:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827338/real 1787827338] req@ffff9a78841db480 x1874671476691072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827338.442453:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476691072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827338.442458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827338.442460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827338.442462:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691072:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827338.442466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691200:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827338.442478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827338.442483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827338/real 1787827338] req@ffff9a78841d9e00 x1874671476691200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827338.442491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476691200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827338.442498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827338.442500:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827338.442502:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691200:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827338.442505:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691328:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827338.442515:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827338.442520:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827338/real 1787827338] req@ffff9a78841d83c0 x1874671476691328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827338.442528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476691328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827338.442534:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827338.442536:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827338.442538:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691328:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827339.465974:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827340.489931:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827341.514006:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827342.537967:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827343.561993:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827343.562003:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827343.562007:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827343.562017:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1711 00000100:00080000:1.0:1787827343.562028:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787827343.562031:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827343.562035:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827343.562077:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476691456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827343.562103:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827343.562105:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827343.562107:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827343.562110:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1711 00000100:00080000:1.0:1787827343.562113:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827343.562117:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827343.562125:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827343.562139:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476691584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827343.562160:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827343.562163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827343.562164:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827343.562168:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1711 00000100:00080000:1.0:1787827343.562170:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827343.562174:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827343.562180:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827343.562185:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476691456:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827343.562192:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476691712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827343.562212:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827343.562214:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827343.562216:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827343.562219:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1711 00000100:00080000:1.0:1787827343.562221:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827343.562223:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827343.562226:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827343.562236:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476691840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827343.562244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827343.562246:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827343.562247:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827343.562251:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1711 00000100:00080000:1.0:1787827343.562254:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827343.562259:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827343.562263:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827343.562272:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476691968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827343.562326:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827343.562411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827343/real 1787827343] req@ffff9a7890b1c3c0 x1874671476691456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827343.562445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476691456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827343.562461:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827343.562466:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827343.562471:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476691456:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827343.562504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691584:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827343.562542:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827343.562566:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827343/real 1787827343] req@ffff9a7890b1e940 x1874671476691584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827343.562591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476691584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827343.562606:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827343.562612:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827343.562616:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691584:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827343.562624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691712:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827343.562651:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827343.562665:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827343/real 1787827343] req@ffff9a7890b1da40 x1874671476691712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827343.562679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476691712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827343.562695:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827343.562699:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827343.562702:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691712:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827343.562709:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691840:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827343.562727:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827343.562737:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827343/real 1787827343] req@ffff9a7890b1f840 x1874671476691840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827343.562749:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476691840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827343.562761:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827343.562764:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827343.562768:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691840:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827343.562775:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691968:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827343.562790:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827343.562825:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827343/real 1787827343] req@ffff9a7890b1f0c0 x1874671476691968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827343.562882:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476691968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827343.562893:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827343.562897:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827343.562901:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476691968:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827344.585958:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827345.610038:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827346.633930:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827347.657994:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827348.681910:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827348.681921:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827348.681925:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827348.681934:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1716 00000100:00080000:1.0:1787827348.681945:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827348.681953:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787827348.681982:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827348.681991:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476692096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827348.682014:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827348.682018:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827348.682019:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827348.682022:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1716 00000100:00080000:1.0:1787827348.682024:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827348.682027:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827348.682035:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827348.682043:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476692224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827348.682052:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476692096:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827348.682053:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827348.682064:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827348.682065:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827348.682069:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1716 00000100:00080000:1.0:1787827348.682071:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827348.682073:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827348.682076:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827348.682087:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476692352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827348.682094:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827348.682096:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827348.682097:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827348.682100:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1716 00000100:00080000:1.0:1787827348.682111:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827348.682115:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827348.682120:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827348.682121:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827348.682138:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476692480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827348.682149:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827348.682153:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827348.682156:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827348.682161:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1716 00000100:00080000:1.0:1787827348.682166:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827348.682172:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827348.682173:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827348/real 1787827348] req@ffff9a7890b1c000 x1874671476692096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787827348.682180:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827348.682195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476692608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827348.682201:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476692096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827348.682216:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827348.682221:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827348.682226:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476692096:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827348.682250:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692224:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827348.682269:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827348.682279:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827348/real 1787827348] req@ffff9a7890b1fc00 x1874671476692224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827348.682287:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476692224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827348.682292:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827348.682294:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827348.682297:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692224:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827348.682300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692352:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827348.682310:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827348.682316:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827348/real 1787827348] req@ffff9a7890b1f0c0 x1874671476692352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827348.682323:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476692352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827348.682328:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827348.682330:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827348.682332:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692352:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827348.682335:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692480:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827348.682344:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827348.682350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827348/real 1787827348] req@ffff9a7890b1f840 x1874671476692480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827348.682356:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476692480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827348.682362:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827348.682365:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827348.682367:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692480:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827348.682370:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692608:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827348.682379:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827348.682384:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827348/real 1787827348] req@ffff9a7890b1da40 x1874671476692608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827348.682391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476692608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827348.682397:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827348.682400:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827348.682402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692608:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827349.705958:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827350.729986:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827351.753992:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827352.777920:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827353.801912:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827353.801946:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827353.801955:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827353.801959:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827353.801968:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1721 00000100:00080000:0.0:1787827353.801977:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827353.801983:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827353.802016:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476692736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827353.802040:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827353.802043:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827353.802044:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827353.802047:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1721 00000100:00080000:0.0:1787827353.802050:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827353.802053:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827353.802061:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827353.802070:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3e580 x1874671476692864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827353.802075:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476692736:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827353.802082:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827353.802086:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827353.802088:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827353.802091:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1721 00000100:00080000:0.0:1787827353.802094:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827353.802096:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827353.802099:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827353.802108:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476692992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827353.802121:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827353.802124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827353.802125:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827353.802128:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1721 00000100:00080000:0.0:1787827353.802131:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827353.802133:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827353.802137:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827353.802146:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476693120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827353.802152:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827353.802154:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827353.802156:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827353.802159:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1721 00000100:00080000:0.0:1787827353.802160:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827353.802163:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827353.802166:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827353.802176:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476693248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827353.802188:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692864:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827353.802217:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692992:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827353.802229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693120:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827353.802245:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693248:192.168.203.140@tcp:8:kworker.0 00000400:00000100:3.0:1787827354.698132:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787827354.698195:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827354.698212:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827354.698255:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827354.698265:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827354.698275:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827354.698286:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827354.698309:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827353/real 1787827354] req@ffff9a7890a3c3c0 x1874671476693248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827354.698339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476693248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827354.698365:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827354.698371:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827354.698376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693248:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827354.698386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827353/real 1787827354] req@ffff9a7890a3e580 x1874671476692864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827354.698401:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3e580 x1874671476692864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827354.698410:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827354.698414:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827354.698417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692864:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827354.698423:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827353/real 1787827354] req@ffff9a7890a3c000 x1874671476692992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827354.698434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476692992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827354.698441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827354.698445:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827354.698448:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476692992:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827354.698455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827353/real 1787827354] req@ffff9a7890a3f480 x1874671476693120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827354.698468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476693120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827354.698478:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827354.698482:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827354.698485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693120:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827354.698490:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827353/real 1787827354] req@ffff9a7890a3de00 x1874671476692736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827354.698505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476692736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827354.698529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827354.698533:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827354.698536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476692736:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787827354.825921:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827355.849918:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827356.873904:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827357.897910:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827358.921930:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827358.921940:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827358.921944:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827358.921961:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1726 00010000:00010000:3.0:1787827358.921961:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827358.921976:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827358.921985:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827358.922026:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476693376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827358.922054:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827358.922057:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827358.922058:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827358.922061:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1726 00000100:00080000:0.0:1787827358.922064:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827358.922066:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827358.922076:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827358.922087:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476693504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827358.922098:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827358.922102:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827358.922105:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827358.922110:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1726 00000100:00080000:0.0:1787827358.922114:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827358.922120:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827358.922127:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827358.922144:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476693632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827358.922146:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476693376:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827358.922169:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827358.922172:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827358.922174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827358.922180:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1726 00000100:00080000:0.0:1787827358.922183:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827358.922186:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827358.922191:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827358.922202:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476693760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827358.922209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827358.922213:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827358.922214:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827358.922217:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1726 00000100:00080000:0.0:1787827358.922220:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827358.922222:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827358.922229:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787827358.922232:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827358.922247:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476693888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827358.922299:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827358/real 1787827358] req@ffff9a7890a3fc00 x1874671476693376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827358.922314:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476693376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827358.922323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827358.922326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827358.922329:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476693376:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827358.922336:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693504:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827358.922353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827358.922364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827358/real 1787827358] req@ffff9a7890a3da40 x1874671476693504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827358.922377:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476693504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827358.922385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827358.922389:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827358.922393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693504:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827358.922420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693632:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827358.922446:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827358.922455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827358/real 1787827358] req@ffff9a7890a3de00 x1874671476693632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827358.922463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476693632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827358.922469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827358.922471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827358.922473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693632:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827358.922477:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693760:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827358.922487:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827358.922493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827358/real 1787827358] req@ffff9a7890a3f480 x1874671476693760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827358.922500:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476693760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827358.922506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827358.922508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827358.922510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693760:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827358.922514:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693888:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827358.922523:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827358.922529:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827358/real 1787827358] req@ffff9a7890a3c000 x1874671476693888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827358.922540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476693888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827358.922547:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827358.922549:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827358.922550:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476693888:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827359.945943:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827360.970032:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827361.993942:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827363.017963:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827364.041960:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827364.041970:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827364.041976:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827364.041987:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1731 00000100:00080000:1.0:1787827364.042007:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787827364.042007:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827364.042018:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827364.042052:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476694016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827364.042074:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827364.042077:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827364.042078:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827364.042081:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1731 00000100:00080000:1.0:1787827364.042083:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827364.042085:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827364.042093:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827364.042101:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476694144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827364.042113:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827364.042115:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827364.042117:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827364.042120:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1731 00000100:00080000:1.0:1787827364.042122:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827364.042125:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476694016:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827364.042127:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827364.042135:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827364.042141:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476694272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827364.042148:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827364.042151:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827364.042152:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827364.042155:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1731 00000100:00080000:1.0:1787827364.042157:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827364.042159:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827364.042163:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827364.042170:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476694400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827364.042177:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827364.042179:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827364.042180:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827364.042183:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1731 00000800:00000100:0.0:1787827364.042187:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827364.042189:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827364.042194:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827364.042198:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827364.042206:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476694528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827364.042234:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9a7890b1e940 x1874671476694016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.042249:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476694016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.042257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.042260:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827364.042263:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476694016:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827364.042268:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694144:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827364.042283:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827364.042290:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9a7890b1c3c0 x1874671476694144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.042298:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476694144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.042304:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.042306:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827364.042308:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694144:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827364.042326:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694272:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827364.042336:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827364.042341:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9a7890b1de00 x1874671476694272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.042353:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476694272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.042359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.042360:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827364.042362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694272:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827364.042365:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694400:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827364.042375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827364.042381:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9a7890b1e580 x1874671476694400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.042388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476694400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.042395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.042396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827364.042398:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694400:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827364.042402:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694528:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827364.042410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827364.042416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9a7890b1f480 x1874671476694528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.042423:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476694528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.042433:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.042435:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827364.042436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694528:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827365.066013:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827366.089949:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827367.114008:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827368.137997:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827369.161929:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.161940:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827369.161944:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787827369.161959:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827369.161965:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1736 00000100:00080000:1.0:1787827369.161977:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.161983:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827369.162019:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476694656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827369.162041:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.162044:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827369.162045:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827369.162048:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1736 00000100:00080000:1.0:1787827369.162050:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.162057:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827369.162065:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827369.162076:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476694784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827369.162086:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476694656:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827369.162087:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.162099:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827369.162100:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827369.162104:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1736 00000100:00080000:1.0:1787827369.162107:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.162109:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827369.162112:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827369.162121:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476694912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827369.162129:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.162131:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827369.162133:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827369.162135:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1736 00000100:00080000:1.0:1787827369.162151:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.162153:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827369.162156:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827369.162164:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827369.162166:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476695040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827369.162174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.162176:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827369.162178:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827369.162181:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1736 00000100:00080000:1.0:1787827369.162183:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.162187:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827369.162192:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827369.162204:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476695168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827369.162226:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9a7890b1cf00 x1874671476694656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.162248:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476694656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.162259:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.162263:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827369.162267:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476694656:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827369.162291:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694784:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827369.162315:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827369.162329:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9a7890b1d680 x1874671476694784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.162342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476694784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.162350:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.162365:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827369.162369:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694784:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827369.162376:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694912:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827369.162396:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827369.162408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9a7890b1f480 x1874671476694912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.162421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476694912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.162429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.162432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827369.162436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476694912:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827369.162442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695040:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827369.162461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827369.162473:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9a7890b1e580 x1874671476695040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.162485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476695040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.162495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.162498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827369.162502:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695040:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827369.162508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695168:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827369.162527:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827369.162539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9a7890b1de00 x1874671476695168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.162552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476695168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.162562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.162565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827369.162569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695168:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827370.186084:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827371.210024:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827372.234053:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827373.257971:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827374.281958:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827374.281971:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827374.281977:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787827374.281989:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827374.281992:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1741 00000100:00080000:0.0:1787827374.282007:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827374.282016:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827374.282057:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3e580 x1874671476695296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827374.282086:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827374.282090:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827374.282092:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827374.282096:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1741 00000100:00080000:0.0:1787827374.282099:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827374.282103:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827374.282113:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827374.282121:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476695424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827374.282140:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476695296:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827374.282143:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827374.282155:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827374.282157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827374.282163:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1741 00000100:00080000:0.0:1787827374.282166:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827374.282171:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827374.282175:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827374.282188:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476695552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827374.282196:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827374.282199:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827374.282201:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827374.282204:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1741 00000100:00080000:0.0:1787827374.282210:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827374.282212:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827374.282217:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827374.282223:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827374.282228:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476695680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827374.282237:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827374.282241:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827374.282243:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827374.282247:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1741 00000100:00080000:0.0:1787827374.282251:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827374.282254:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827374.282257:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827374.282266:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476695808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827374.282304:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9a7890a3e580 x1874671476695296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827374.282321:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3e580 x1874671476695296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827374.282329:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827374.282332:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827374.282335:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476695296:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827374.282356:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695424:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827374.282375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827374.282383:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9a7890a3c3c0 x1874671476695424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827374.282391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476695424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827374.282396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827374.282398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827374.282400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695424:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827374.282406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695552:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827374.282416:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827374.282422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9a7890a3c000 x1874671476695552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827374.282430:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476695552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827374.282434:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827374.282436:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827374.282438:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695552:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827374.282442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695680:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827374.282452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827374.282458:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9a7890a3f480 x1874671476695680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827374.282465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476695680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827374.282471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827374.282473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827374.282475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695680:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827374.282478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695808:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827374.282488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827374.282494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9a7890a3de00 x1874671476695808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827374.282501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476695808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827374.282507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827374.282509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827374.282511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476695808:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827375.306006:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827376.330000:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827377.353914:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827378.377988:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827379.401891:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827379.401899:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827379.401902:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827379.401911:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1746 00000100:00080000:3.0:1787827379.401933:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827379.401940:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827379.401975:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476695936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827379.402013:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827379.402017:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827379.402019:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827379.402022:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1746 00000100:00080000:3.0:1787827379.402024:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827379.402027:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827379.402037:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827379.402048:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476696064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827379.402057:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827379.402061:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787827379.402063:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476695936:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827379.402064:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827379.402075:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1746 00000100:00080000:3.0:1787827379.402078:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827379.402081:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827379.402084:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827379.402103:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476696192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827379.402109:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827379.402112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827379.402114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827379.402116:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1746 00000100:00080000:3.0:1787827379.402126:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827379.402129:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827379.402133:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827379.402136:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827379.402145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476696320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827379.402164:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827379.402166:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827379.402167:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827379.402170:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1746 00000100:00080000:3.0:1787827379.402172:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827379.402175:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827379.402178:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827379.402189:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476696448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827379.402194:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9a78a6775a40 x1874671476695936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.402210:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476695936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.402220:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.402224:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827379.402228:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476695936:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787827379.402242:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827379.402258:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696064:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827379.402287:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827379.402296:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9a78a6774000 x1874671476696064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.402304:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476696064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.402309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.402311:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827379.402314:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696064:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827379.402318:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696192:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827379.402328:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827379.402335:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9a78a6776580 x1874671476696192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.402348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476696192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.402355:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.402370:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827379.402373:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696192:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827379.402380:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696320:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827379.402399:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827379.402408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9a78a6776940 x1874671476696320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.402416:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476696320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.402423:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.402425:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827379.402427:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696320:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827379.402431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696448:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827379.402442:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827379.402448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9a78a67770c0 x1874671476696448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.402455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476696448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.402461:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.402463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827379.402465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696448:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827380.425928:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827381.450014:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827382.474006:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827383.497918:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827384.521944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827384.521956:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827384.521963:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827384.521975:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1752 00000100:00080000:3.0:1787827384.521989:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827384.521998:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827384.522048:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476696576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827384.522082:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827384.522086:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827384.522088:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827384.522091:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1752 00000100:00080000:3.0:1787827384.522094:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827384.522098:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827384.522111:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827384.522124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476696704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827384.522135:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827384.522138:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827384.522140:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827384.522145:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1752 00000100:00080000:3.0:1787827384.522152:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827384.522154:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476696576:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827384.522158:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827384.522174:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827384.522192:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476696832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827384.522208:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827384.522211:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827384.522213:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827384.522217:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1752 00000100:00080000:3.0:1787827384.522234:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827384.522238:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827384.522244:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827384.522250:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827384.522260:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476696960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827384.522273:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827384.522276:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827384.522278:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827384.522283:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1752 00000100:00080000:3.0:1787827384.522287:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827384.522292:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827384.522298:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827384.522308:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476697088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827384.522318:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9a78a6774780 x1874671476696576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827384.522338:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476696576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827384.522348:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827384.522351:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827384.522356:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476696576:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827384.522387:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696704:192.168.203.140@tcp:38:kworker.0 00010000:00010000:3.0:1787827384.522409:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:0.0:1787827384.522424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827384.522437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9a78a67761c0 x1874671476696704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827384.522446:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476696704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827384.522456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827384.522458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827384.522478:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696704:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827384.522483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696832:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827384.522496:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827384.522503:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9a78a6776d00 x1874671476696832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827384.522526:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476696832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827384.522531:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827384.522534:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827384.522536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696832:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827384.522541:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696960:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827384.522556:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827384.522569:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9a78a6775680 x1874671476696960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827384.522576:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476696960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827384.522582:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827384.522584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827384.522587:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476696960:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827384.522591:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697088:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827384.522601:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827384.522606:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9a78a6777840 x1874671476697088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827384.522613:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476697088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827384.522619:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827384.522621:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827384.522623:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697088:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827385.546009:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827386.570045:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827387.594029:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827388.617984:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827389.641974:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827389.641985:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:3.0:1787827389.641988:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827389.641990:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827389.642001:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1757 00000100:00080000:1.0:1787827389.642011:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827389.642019:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827389.642055:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476697216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827389.642083:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827389.642087:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827389.642089:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827389.642093:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1757 00000100:00080000:1.0:1787827389.642098:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827389.642102:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827389.642112:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827389.642126:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476697344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827389.642138:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476697216:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827389.642153:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827389.642157:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827389.642160:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827389.642165:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1757 00000100:00080000:1.0:1787827389.642169:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827389.642174:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827389.642179:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827389.642192:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476697472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827389.642203:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827389.642206:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827389.642208:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827389.642212:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1757 00000100:00080000:1.0:1787827389.642216:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827389.642221:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827389.642226:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827389.642240:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476697600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827389.642250:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827389.642253:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827389.642255:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827389.642260:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697344:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787827389.642265:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1757 00000100:00080000:1.0:1787827389.642272:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827389.642277:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827389.642283:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827389.642296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476697728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827389.642305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697472:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827389.642338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697600:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827389.642366:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697728:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787827390.666212:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827391.690122:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827392.714266:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827392.714365:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787827392.714476:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827392.714498:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827392.714596:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827392.714608:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827392.714619:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827392.714631:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827392.714757:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827392] req@ffff9a7890b1f0c0 x1874671476697728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827392.714795:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476697728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827392.714813:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827392.714820:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827392.714826:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697728:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827392.715286:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827392] req@ffff9a7890b1e940 x1874671476697344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827392.715306:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476697344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827392.715314:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827392.715317:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827392.715321:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697344:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827392.715326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827392] req@ffff9a7890b1da40 x1874671476697472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827392.715341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476697472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827392.715347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827392.715350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827392.715352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697472:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827392.715357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827392] req@ffff9a7890b1f840 x1874671476697600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827392.715365:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476697600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827392.715372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827392.715375:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827392.715377:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697600:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827392.715382:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827392] req@ffff9a7890b1c3c0 x1874671476697216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827392.715390:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476697216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827392.715396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827392.715398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827392.715401:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476697216:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787827393.738031:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827394.762035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827394.762061:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827394.762068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827394.762086:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1762 00000100:00080000:1.0:1787827394.762104:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827394.762129:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827394.762194:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476697856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827394.762212:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827394.762234:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827394.762239:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827394.762242:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827394.762246:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1762 00000100:00080000:1.0:1787827394.762250:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827394.762254:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827394.762266:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827394.762280:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476697984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827394.762291:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827394.762295:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827394.762297:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827394.762305:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1762 00000100:00080000:1.0:1787827394.762308:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827394.762309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476697856:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827394.762311:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827394.762320:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827394.762331:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476698112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827394.762341:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827394.762346:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827394.762358:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827394.762364:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1762 00000100:00080000:1.0:1787827394.762369:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827394.762374:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827394.762382:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827394.762399:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476698240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827394.762412:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827394.762417:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827394.762420:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827394.762425:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1762 00000100:00080000:1.0:1787827394.762430:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787827394.762456:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827394.762459:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827394.762469:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827394.762490:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476698368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827394.762550:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827394/real 1787827394] req@ffff9a7890b1c3c0 x1874671476697856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827394.762580:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476697856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827394.762613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827394.762619:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827394.762625:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476697856:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827394.762663:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697984:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827394.762696:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827394.762714:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827394/real 1787827394] req@ffff9a7890b1f840 x1874671476697984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827394.762731:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476697984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827394.762744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827394.762750:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827394.762754:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476697984:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827394.762763:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698112:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827394.762793:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827394.762809:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827394/real 1787827394] req@ffff9a7890b1da40 x1874671476698112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827394.762826:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476698112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827394.762836:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827394.762841:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827394.762907:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698112:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827394.762918:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698240:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827394.762947:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827394.762974:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827394/real 1787827394] req@ffff9a7890b1e940 x1874671476698240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827394.762992:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476698240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827394.763008:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827394.763012:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827394.763017:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698240:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827394.763025:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698368:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827394.763053:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827394.763069:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827394/real 1787827394] req@ffff9a7890b1f0c0 x1874671476698368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827394.763085:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476698368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827394.763098:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827394.763102:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827394.763106:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698368:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827395.786202:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827396.810028:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827397.834079:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827398.858056:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827399.881905:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827399.881914:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827399.881919:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827399.881928:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1767 00000100:00080000:1.0:1787827399.881939:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827399.881946:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827399.881959:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827399.881980:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476698496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827399.882000:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827399.882002:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827399.882004:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827399.882010:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1767 00000100:00080000:1.0:1787827399.882013:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827399.882016:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827399.882024:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827399.882032:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476698624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827399.882042:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476698496:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827399.882048:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827399.882053:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827399.882055:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827399.882060:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1767 00000100:00080000:1.0:1787827399.882061:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827399.882063:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827399.882066:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827399.882076:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476698752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827399.882083:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827399.882086:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827399.882087:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827399.882090:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1767 00000100:00080000:1.0:1787827399.882092:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827399.882094:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827399.882097:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827399.882104:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476698880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827399.882110:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827399.882113:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827399.882114:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827399.882117:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1767 00000100:00080000:1.0:1787827399.882185:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827399.882188:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827399.882191:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827399.882198:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476699008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827399.882212:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827399.882283:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827399/real 1787827399] req@ffff9a7890b1fc00 x1874671476698496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827399.882304:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476698496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827399.882316:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827399.882320:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827399.882325:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476698496:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827399.882372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698624:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827399.882398:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827399.882429:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827399/real 1787827399] req@ffff9a7890b1c000 x1874671476698624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827399.882441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476698624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827399.882454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827399.882457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827399.882460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698624:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827399.882466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698752:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827399.882484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827399.882492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827399/real 1787827399] req@ffff9a7890b1f0c0 x1874671476698752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827399.882502:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476698752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827399.882508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827399.882511:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827399.882513:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698752:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827399.882519:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698880:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827399.882533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827399.882541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827399/real 1787827399] req@ffff9a7890b1e940 x1874671476698880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827399.882550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476698880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827399.882559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827399.882561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827399.882564:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476698880:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827399.882570:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699008:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827399.882582:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827399.882588:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827399/real 1787827399] req@ffff9a7890b1da40 x1874671476699008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827399.882597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476699008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827399.882605:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827399.882608:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827399.882610:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699008:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827400.905994:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827401.930015:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827402.954041:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827403.978169:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827405.001972:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827405.001982:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827405.001987:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827405.001998:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1772 00000100:00080000:1.0:1787827405.002009:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827405.002017:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827405.002057:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827405.002059:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476699136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827405.002090:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827405.002094:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827405.002096:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827405.002100:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1772 00000100:00080000:1.0:1787827405.002103:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827405.002106:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827405.002117:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827405.002149:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476699264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827405.002172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827405.002180:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476699136:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827405.002180:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827405.002186:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827405.002198:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1772 00000100:00080000:1.0:1787827405.002201:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827405.002206:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827405.002211:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827405.002234:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476699392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827405.002245:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827405.002250:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827405.002252:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827405.002267:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1772 00000100:00080000:1.0:1787827405.002271:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827405.002280:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827405.002281:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827405.002292:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827405.002308:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476699520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827405.002319:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827405.002341:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827405.002343:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827405.002349:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1772 00000100:00080000:1.0:1787827405.002355:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827405.002378:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827405.002385:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827405.002408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476699648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827405.002419:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9a7890b1f840 x1874671476699136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827405.002442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476699136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827405.002455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827405.002460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827405.002465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476699136:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827405.002493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699264:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827405.002517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827405.002549:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9a7890b1c3c0 x1874671476699264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827405.002563:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476699264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827405.002572:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827405.002575:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827405.002579:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699264:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827405.002585:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699392:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827405.002603:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827405.002612:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9a7890b1de00 x1874671476699392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827405.002624:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476699392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827405.002632:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827405.002635:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827405.002638:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699392:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827405.002644:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699520:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827405.002659:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827405.002668:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9a7890b1e580 x1874671476699520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827405.002688:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476699520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827405.002698:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827405.002701:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827405.002704:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699520:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827405.002709:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699648:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827405.002724:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827405.002733:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9a7890b1f480 x1874671476699648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827405.002744:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476699648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827405.002754:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827405.002756:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827405.002759:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699648:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827406.026095:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827407.049983:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827408.073974:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827409.098011:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827410.122045:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827410.122081:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827410.122089:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827410.122097:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827410.122104:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1777 00000100:00080000:3.0:1787827410.122119:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827410.122129:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827410.122180:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476699776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827410.122224:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827410.122232:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827410.122236:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827410.122243:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1777 00000100:00080000:3.0:1787827410.122251:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827410.122259:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827410.122272:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827410.122278:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476699776:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827410.122297:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476699904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827410.122316:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827410.122320:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827410.122323:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827410.122374:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1777 00000100:00080000:3.0:1787827410.122383:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827410.122389:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827410.122397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787827410.122402:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827410.122426:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476700032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827410.122441:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827410.122446:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827410.122449:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827410.122454:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1777 00000100:00100000:0.0:1787827410.122469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9a78a6775e00 x1874671476699776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827410.122472:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827410.122487:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827410.122500:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827410.122526:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476700160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827410.122542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476699776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827410.122549:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827410.122565:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827410.122572:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827410.122579:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827410.122581:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1777 00000100:00080000:3.0:1787827410.122589:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827410.122591:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787827410.122605:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827410.122606:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476699776:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787827410.122620:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827410.122639:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476700288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827410.122652:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699904:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827410.122704:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827410.122734:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9a78a6777c00 x1874671476699904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827410.122750:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476699904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827410.122760:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827410.122764:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827410.122768:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476699904:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827410.122775:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700032:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827410.122818:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827410.122898:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9a78a67743c0 x1874671476700032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827410.122931:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476700032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827410.122949:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827410.122953:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827410.122957:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700032:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827410.122965:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700160:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827410.122993:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827410.123006:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9a78a6774f00 x1874671476700160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827410.123020:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476700160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827410.123030:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827410.123034:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827410.123037:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700160:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827410.123058:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700288:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827410.123076:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827410.123087:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9a78a6777840 x1874671476700288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827410.123100:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476700288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827410.123111:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827410.123114:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827410.123120:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700288:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827411.146116:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827412.170136:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827413.194135:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827414.217968:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827415.241954:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827415.241968:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827415.241974:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827415.241990:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1782 00000100:00080000:1.0:1787827415.242005:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827415.242015:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827415.242034:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827415.242081:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476700416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827415.242136:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827415.242147:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827415.242151:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827415.242157:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1782 00000100:00080000:1.0:1787827415.242165:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827415.242174:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827415.242189:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827415.242224:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476700416:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827415.242239:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476700544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827415.242258:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827415.242263:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827415.242266:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827415.242272:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1782 00000100:00080000:1.0:1787827415.242278:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827415.242283:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827415.242291:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827415.242333:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476700672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827415.242364:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827415.242368:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827415.242373:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787827415.242376:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827415.242381:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1782 00000100:00080000:1.0:1787827415.242388:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827415.242393:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827415.242401:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827415.242413:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476700800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827415.242424:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827415.242428:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827415.242430:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827415.242435:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1782 00000100:00080000:1.0:1787827415.242440:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827415.242443:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827415.242448:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827415.242455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9a7890b1d680 x1874671476700416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827415.242465:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476700928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827415.242535:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476700416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.242557:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.242565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827415.242572:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476700416:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827415.242608:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700544:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827415.242671:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827415.242699:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9a7890b1cf00 x1874671476700544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827415.242726:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476700544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.242744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.242767:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827415.242773:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700544:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827415.242783:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700672:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827415.242815:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827415.242834:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9a7890b1f480 x1874671476700672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827415.242854:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476700672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.242868:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.242874:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827415.242879:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700672:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827415.242890:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700800:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827415.242985:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827415.243006:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9a7890b1e580 x1874671476700800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827415.243027:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476700800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.243043:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.243065:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827415.243072:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700800:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827415.243083:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700928:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827415.243120:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827415.243142:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9a7890b1de00 x1874671476700928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827415.243193:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476700928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827415.243211:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827415.243216:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827415.243240:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476700928:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827416.266017:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827417.290091:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827418.314001:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827419.338045:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827420.361904:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827420.361913:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827420.361918:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827420.361928:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1787 00000100:00080000:1.0:1787827420.361939:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827420.361945:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827420.361952:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827420.361981:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476701056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827420.362013:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827420.362017:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827420.362020:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827420.362027:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1787 00000100:00080000:1.0:1787827420.362033:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827420.362038:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827420.362044:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476701056:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787827420.362052:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827420.362069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476701184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827420.362080:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827420.362083:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827420.362085:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827420.362089:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1787 00000100:00080000:1.0:1787827420.362102:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787827420.362109:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827420.362109:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827420.362116:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827420.362128:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476701312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827420.362137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827420.362140:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827420.362142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827420.362146:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1787 00000100:00080000:1.0:1787827420.362150:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827420.362153:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827420.362159:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9a7890b1c3c0 x1874671476701056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787827420.362159:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827420.362175:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476701440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827420.362183:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476701056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827420.362221:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827420.362224:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827420.362224:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827420.362229:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827420.362233:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787827420.362236:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1787 00000100:00080000:1.0:1787827420.362240:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827420.362242:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476701056:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827420.362246:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827420.362253:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827420.362281:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476701568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827420.362291:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701184:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827420.362319:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827420.362332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9a7890b1f840 x1874671476701184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827420.362349:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476701184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827420.362356:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827420.362359:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827420.362361:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701184:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827420.362366:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701312:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827420.362379:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827420.362386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9a7890b1da40 x1874671476701312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827420.362395:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476701312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827420.362401:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827420.362403:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827420.362406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701312:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827420.362410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701440:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827420.362422:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827420.362428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9a7890b1e940 x1874671476701440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827420.362437:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476701440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827420.362450:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827420.362452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827420.362455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701440:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827420.362468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701568:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827420.362490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827420.362499:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9a7890b1f0c0 x1874671476701568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827420.362528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476701568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827420.362541:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827420.362545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827420.362549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701568:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827421.386024:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827422.409929:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827423.433943:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827424.457983:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827425.481996:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827425.482014:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827425.482021:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827425.482039:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1792 00000100:00080000:1.0:1787827425.482056:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827425.482069:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827425.482080:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827425.482125:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476701696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827425.482175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827425.482183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827425.482187:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827425.482194:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1792 00000100:00080000:1.0:1787827425.482203:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827425.482214:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827425.482261:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827425.482273:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476701696:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827425.482296:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476701824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827425.482324:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827425.482332:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827425.482335:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827425.482345:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1792 00000100:00080000:1.0:1787827425.482353:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787827425.482362:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827425.482369:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827425.482383:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827425.482412:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476701952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827425.482434:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827425] req@ffff9a7890b1c000 x1874671476701696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827425.482437:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827425.482455:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827425.482464:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827425.482479:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1792 00000100:00100000:0.0:1787827425.482482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476701696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827425.482493:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827425.482509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827425.482512:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787827425.482523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827425.482532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476701696:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787827425.482534:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827425.482570:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701824:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827425.482575:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476702080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827425.482604:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1787827425.482609:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827425.482620:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827425.482627:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827425.482634:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827425] req@ffff9a7890b1fc00 x1874671476701824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827425.482644:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1792 00000100:00080000:1.0:1787827425.482662:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827425.482671:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476701824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827425.482682:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787827425.482691:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827425.482701:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000008:00100000:1.0:1787827425.482701:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827425.482715:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701824:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827425.482728:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701952:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827425.482747:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476702208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827425.482770:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827425.482820:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827425] req@ffff9a7890b1f0c0 x1874671476701952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827425.482892:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476701952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827425.482903:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827425.482907:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827425.482911:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476701952:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827425.482933:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702080:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827425.482964:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827425.482979:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827425] req@ffff9a7890b1e940 x1874671476702080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827425.482993:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476702080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827425.483005:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827425.483023:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827425.483027:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702080:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827425.483035:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702208:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827425.483056:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827425.483068:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827425] req@ffff9a7890b1da40 x1874671476702208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827425.483082:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476702208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827425.483093:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827425.483096:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827425.483100:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702208:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827426.506097:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827427.530141:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827428.554063:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827429.578061:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827430.602048:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.602063:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:0.0:1787827430.602069:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827430.602070:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.602085:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1798 00000100:00080000:2.0:1787827430.602097:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.602106:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827430.602151:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476702336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827430.602175:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.602178:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827430.602180:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.602183:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1798 00000100:00080000:2.0:1787827430.602185:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.602200:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827430.602211:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827430.602223:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476702464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827430.602224:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476702336:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827430.602239:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.602241:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827430.602243:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.602247:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1798 00000100:00080000:2.0:1787827430.602250:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.602252:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827430.602256:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827430.602267:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476702592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827430.602274:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.602277:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827430.602279:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.602282:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1798 00000100:00080000:2.0:1787827430.602292:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.602294:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827430.602298:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827430.602303:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827430.602309:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476702720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827430.602317:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.602320:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827430.602321:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.602324:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1798 00000100:00080000:2.0:1787827430.602327:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.602330:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827430.602332:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827430.602340:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476702848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827430.602364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9a78841dbc00 x1874671476702336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827430.602380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476702336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827430.602391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827430.602396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827430.602401:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476702336:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827430.602430:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702464:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827430.602454:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827430.602465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9a78841da1c0 x1874671476702464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827430.602474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476702464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827430.602480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827430.602483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827430.602485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702464:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827430.602489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702592:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827430.602500:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827430.602506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9a78841db0c0 x1874671476702592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827430.602514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476702592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827430.602524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827430.602527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827430.602528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702592:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827430.602532:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702720:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827430.602542:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827430.602548:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9a78841d9a40 x1874671476702720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827430.602554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476702720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827430.602562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827430.602564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827430.602566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702720:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827430.602571:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702848:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827430.602580:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827430.602585:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9a78841d8780 x1874671476702848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827430.602592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476702848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827430.602599:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827430.602601:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827430.602604:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476702848:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827431.626129:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827432.650088:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827433.674035:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827434.697982:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827435.721965:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827435.721979:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827435.721986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827435.721989:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827435.722000:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1803 00000100:00080000:2.0:1787827435.722015:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827435.722025:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827435.722077:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476702976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827435.722108:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827435.722112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827435.722115:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827435.722120:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1803 00000100:00080000:2.0:1787827435.722125:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827435.722131:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827435.722143:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827435.722166:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476702976:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827435.722177:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476703104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827435.722193:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827435.722197:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827435.722200:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827435.722205:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1803 00000100:00080000:2.0:1787827435.722210:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827435.722215:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827435.722222:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827435.722235:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476703232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827435.722246:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1787827435.722248:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787827435.722255:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827435.722258:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827435.722267:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1803 00000100:00080000:2.0:1787827435.722273:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827435.722278:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827435.722284:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827435.722299:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476703360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827435.722308:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827435.722315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827435/real 1787827435] req@ffff9a78841d8780 x1874671476702976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827435.722324:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827435.722329:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827435.722341:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1803 00000100:00100000:0.0:1787827435.722348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476702976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827435.722350:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827435.722362:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827435.722371:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787827435.722377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827435.722387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827435.722389:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476703488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827435.722399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476702976:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827435.722451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703104:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827435.722480:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827435.722495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827435/real 1787827435] req@ffff9a78841d9a40 x1874671476703104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827435.722508:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476703104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827435.722516:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827435.722520:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827435.722523:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703104:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827435.722533:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703232:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827435.722552:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827435.722564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827435/real 1787827435] req@ffff9a78841db0c0 x1874671476703232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827435.722575:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476703232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827435.722583:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827435.722587:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827435.722590:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703232:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827435.722597:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703360:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827435.722617:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827435.722628:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827435/real 1787827435] req@ffff9a78841da1c0 x1874671476703360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827435.722640:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476703360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827435.722651:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827435.722655:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827435.722658:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703360:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827435.722665:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703488:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827435.722692:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827435.722704:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827435/real 1787827435] req@ffff9a78841dbc00 x1874671476703488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827435.722715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476703488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827460 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827435.722725:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827435.722728:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827435.722739:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703488:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827436.746081:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827437.769965:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827438.794065:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827439.817975:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827440.841939:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827440.841960:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827440.841963:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827440.841973:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1808 00000100:00080000:2.0:1787827440.841983:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827440.841990:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827440.842010:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827440.842026:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476703616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827440.842050:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827440.842054:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827440.842056:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827440.842059:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1808 00000100:00080000:2.0:1787827440.842061:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827440.842064:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827440.842072:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827440.842080:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476703744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827440.842089:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827440.842092:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827440.842093:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827440.842096:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1808 00000100:00080000:2.0:1787827440.842099:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827440.842101:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827440.842105:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827440.842113:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476703616:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827440.842115:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476703872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827440.842144:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827440.842147:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827440.842149:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827440.842152:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1808 00000100:00080000:2.0:1787827440.842156:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827440.842160:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827440.842166:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827440.842176:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476704000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827440.842183:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827440.842186:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827440.842187:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787827440.842192:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787827440.842196:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1808 00000100:00080000:2.0:1787827440.842201:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827440.842204:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827440.842207:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827440.842217:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476704128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827440.842244:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827440/real 1787827440] req@ffff9a78841d83c0 x1874671476703616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827440.842270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476703616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827440.842279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827440.842282:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827440.842285:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476703616:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827440.842292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703744:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827440.842307:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827440.842315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827440/real 1787827440] req@ffff9a78841d9e00 x1874671476703744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827440.842341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476703744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827440.842347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827440.842350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827440.842352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703744:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827440.842371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703872:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827440.842384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827440.842392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827440/real 1787827440] req@ffff9a78841db480 x1874671476703872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827440.842401:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476703872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827440.842406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827440.842409:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827440.842411:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476703872:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827440.842415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704000:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827440.842426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827440.842432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827440/real 1787827440] req@ffff9a78841d8000 x1874671476704000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827440.842440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476704000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827440.842452:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827440.842454:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827440.842456:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704000:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827440.842460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704128:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827440.842470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827440.842476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827440/real 1787827440] req@ffff9a78841da580 x1874671476704128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827440.842483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476704128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827465 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827440.842491:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827440.842493:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827440.842495:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704128:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827441.865993:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827442.890095:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827443.914121:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827444.938056:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827445.962001:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827445.962013:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827445.962019:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827445.962031:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1813 00000100:00080000:2.0:1787827445.962043:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827445.962052:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827445.962147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476704256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827445.962187:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827445.962190:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827445.962197:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827445.962200:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827445.962204:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1813 00000100:00080000:2.0:1787827445.962211:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827445.962217:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827445.962230:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827445.962248:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476704384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827445.962264:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827445.962268:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827445.962270:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827445.962278:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1813 00000100:00100000:1.0:1787827445.962281:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476704256:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827445.962282:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827445.962292:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827445.962302:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827445.962316:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476704512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827445.962327:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827445.962332:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827445.962334:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827445.962355:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1813 00000100:00080000:2.0:1787827445.962374:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827445.962379:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827445.962385:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827445.962389:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827445.962422:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476704640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827445.962446:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827445.962450:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827445.962453:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827445.962458:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1813 00000100:00080000:2.0:1787827445.962464:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827445.962470:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827445.962476:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827445.962484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827445/real 1787827445] req@ffff9a78841d8b40 x1874671476704256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827445.962497:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476704768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827445.962521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476704256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827445.962548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827445.962570:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827445.962575:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476704256:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827445.962610:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704384:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827445.962636:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827445.962649:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827445/real 1787827445] req@ffff9a78841db840 x1874671476704384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827445.962669:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476704384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827445.962677:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827445.962680:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827445.962684:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704384:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827445.962690:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704512:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827445.962705:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827445.962727:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827445/real 1787827445] req@ffff9a78841d92c0 x1874671476704512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827445.962739:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476704512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827445.962747:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827445.962750:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827445.962754:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704512:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827445.962762:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704640:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827445.962791:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827445.962810:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827445/real 1787827445] req@ffff9a78841d9680 x1874671476704640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827445.962878:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476704640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827445.962920:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827445.962927:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827445.962933:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704640:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827445.962970:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704768:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827445.963014:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827445.963031:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827445/real 1787827445] req@ffff9a78841da940 x1874671476704768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827445.963045:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476704768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827470 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827445.963056:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827445.963060:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827445.963064:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476704768:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827446.986134:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827448.010055:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827449.033943:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827450.057968:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827451.081909:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.081920:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827451.081924:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827451.081935:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1818 00000100:00080000:2.0:1787827451.081945:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.081951:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827451.081974:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827451.081991:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476704896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827451.082012:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.082015:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827451.082016:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827451.082019:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1818 00000100:00080000:2.0:1787827451.082021:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.082025:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827451.082034:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827451.082043:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476704896:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827451.082060:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476705024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827451.082072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.082075:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827451.082076:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827451.082080:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1818 00000100:00080000:2.0:1787827451.082082:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.082085:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827451.082089:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827451.082104:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476705152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827451.082112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.082114:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827451.082116:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1787827451.082119:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787827451.082123:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1818 00000100:00080000:2.0:1787827451.082128:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.082131:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827451.082135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827451.082143:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476705280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827451.082150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827451.082152:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827451.082154:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827451.082157:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1818 00000100:00080000:2.0:1787827451.082160:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827451.082163:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827451.082166:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9a78841da940 x1874671476704896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787827451.082168:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827451.082182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476705408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827451.082188:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476704896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827451.082206:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827451.082209:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827451.082213:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476704896:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827451.082230:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705024:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827451.082245:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827451.082253:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9a78841d9680 x1874671476705024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827451.082261:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476705024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827451.082266:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827451.082268:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827451.082270:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705024:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827451.082273:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705152:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827451.082283:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827451.082289:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9a78841d92c0 x1874671476705152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827451.082296:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476705152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827451.082301:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827451.082303:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827451.082305:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705152:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827451.082309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705280:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827451.082319:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827451.082325:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9a78841db840 x1874671476705280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827451.082331:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476705280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827451.082338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827451.082339:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827451.082341:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705280:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827451.082345:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705408:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827451.082354:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827451.082359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827451/real 1787827451] req@ffff9a78841d8b40 x1874671476705408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827451.082366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476705408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827451.082372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827451.082373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827451.082375:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705408:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827452.106029:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827453.129977:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827454.154021:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827455.178073:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827456.201996:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827456.202007:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827456.202022:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827456.202032:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1823 00000100:00080000:1.0:1787827456.202052:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827456.202059:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827456.202092:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827456.202098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476705536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827456.202124:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827456.202127:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827456.202128:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827456.202131:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1823 00000100:00080000:1.0:1787827456.202133:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827456.202136:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827456.202144:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827456.202154:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476705664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827456.202161:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827456.202164:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827456.202165:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827456.202173:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1823 00000100:00100000:0.0:1787827456.202174:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476705536:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827456.202176:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827456.202185:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827456.202194:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827456.202214:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476705792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827456.202220:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827456.202222:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827456.202223:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827456.202227:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1823 00000100:00080000:1.0:1787827456.202229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827456.202231:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827456.202236:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827456.202249:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476705920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827456.202255:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827456.202257:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827456.202258:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827456.202261:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1823 00000100:00080000:1.0:1787827456.202273:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827456.202275:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827456.202280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827456.202288:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476706048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827456.202390:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705664:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827456.202433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705792:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827456.202453:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705920:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827456.202489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706048:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827457.226023:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827458.250116:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827459.274147:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827459.274185:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787827459.274228:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827459.274244:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827459.274282:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827459.274295:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827459.274308:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827459.274318:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827459] req@ffff9a7890b1c3c0 x1874671476705664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:0.0:1787827459.274319:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827459.274358:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476705664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827459.274371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827459.274376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827459.274389:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705664:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827459.274398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827459] req@ffff9a7890b1de00 x1874671476705792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827459.274413:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476705792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827459.274422:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827459.274426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827459.274430:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705792:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827459.274438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827459] req@ffff9a7890b1e580 x1874671476705920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827459.274451:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476705920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827459.274462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827459.274466:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827459.274469:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476705920:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827459.274477:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827459] req@ffff9a7890b1f480 x1874671476706048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827459.274490:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476706048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827459.274500:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827459.274504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827459.274507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706048:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827459.274514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827456/real 1787827459] req@ffff9a7890b1f840 x1874671476705536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827459.274536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476705536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827459.274545:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827459.274548:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827459.274552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476705536:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787827460.298062:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827461.321928:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827461.321962:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827461.321967:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827461.321988:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1828 00000100:00080000:1.0:1787827461.321999:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827461.322007:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827461.322041:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476706176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827461.322071:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827461.322078:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827461.322081:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827461.322084:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827461.322087:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1828 00000100:00080000:1.0:1787827461.322092:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827461.322107:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827461.322118:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827461.322133:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476706304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827461.322146:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827461.322149:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827461.322151:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827461.322155:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1828 00000100:00080000:1.0:1787827461.322160:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827461.322168:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827461.322171:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476706176:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787827461.322174:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827461.322196:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476706432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827461.322206:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827461.322210:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827461.322212:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827461.322217:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1828 00000100:00080000:1.0:1787827461.322221:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827461.322225:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827461.322231:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827461.322244:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476706560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827461.322254:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827461.322257:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827461.322259:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827461.322264:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1828 00000100:00080000:1.0:1787827461.322268:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827461.322273:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827461.322277:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787827461.322285:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827461.322293:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476706688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827461.322365:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9a7890b1f840 x1874671476706176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827461.322391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476706176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827461.322407:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827461.322413:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827461.322418:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476706176:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827461.322460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706304:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827461.322490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827461.322506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9a7890b1f480 x1874671476706304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827461.322521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476706304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827461.322538:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827461.322542:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827461.322546:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706304:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827461.322554:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706432:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827461.322577:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827461.322590:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9a7890b1e580 x1874671476706432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827461.322604:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476706432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827461.322613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827461.322617:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827461.322621:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706432:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827461.322635:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706560:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827461.322657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827461.322669:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9a7890b1de00 x1874671476706560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827461.322683:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476706560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827461.322695:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827461.322699:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827461.322703:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706560:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827461.322710:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706688:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827461.322732:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827461.322744:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827461/real 1787827461] req@ffff9a7890b1c3c0 x1874671476706688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827461.322757:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476706688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827461.322767:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827461.322777:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827461.322781:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706688:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827462.346152:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827463.369992:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827464.393983:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827465.417998:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827466.442025:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787827466.442028:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827466.442039:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827466.442045:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827466.442059:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1833 00000100:00080000:1.0:1787827466.442073:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827466.442082:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827466.442149:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476706816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827466.442206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827466.442211:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827466.442214:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827466.442218:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1833 00000100:00080000:1.0:1787827466.442224:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827466.442236:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827466.442251:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827466.442291:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476706944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827466.442308:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827466.442314:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827466.442317:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827466.442325:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1833 00000100:00080000:1.0:1787827466.442333:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827466.442336:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476706816:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827466.442362:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827466.442374:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827466.442396:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476707072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827466.442408:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827466.442412:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827466.442414:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827466.442419:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1833 00000100:00080000:1.0:1787827466.442443:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827466.442447:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827466.442454:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787827466.442461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827466.442474:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476707200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827466.442493:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827466.442496:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827466.442498:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827466.442503:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1833 00000100:00080000:1.0:1787827466.442507:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827466.442511:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827466.442517:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827466.442530:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476707328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827466.442539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff9a7890b1cf00 x1874671476706816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.442563:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476706816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.442576:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.442581:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827466.442586:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476706816:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827466.442594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706944:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827466.442617:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827466.442629:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff9a7890b1d680 x1874671476706944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.442682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476706944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.442691:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.442694:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827466.442698:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476706944:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827466.442729:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707072:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827466.442751:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827466.442762:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff9a7890b1c3c0 x1874671476707072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.442774:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476707072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.442782:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.442784:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827466.442803:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707072:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827466.442809:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707200:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827466.442826:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827466.442835:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff9a7890b1de00 x1874671476707200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.442907:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476707200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.442921:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.442925:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827466.442929:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707200:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827466.442936:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707328:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827466.442998:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827466.443017:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827466/real 1787827466] req@ffff9a7890b1e580 x1874671476707328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827466.443031:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476707328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827466.443044:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827466.443047:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827466.443069:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707328:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827467.465972:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827468.489997:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827469.513999:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827470.537960:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827471.562041:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827471.562049:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827471.562053:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827471.562063:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827471.562063:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1839 00000100:00080000:3.0:1787827471.562074:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827471.562079:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827471.562111:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476707456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827471.562134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827471.562137:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827471.562138:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827471.562141:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1839 00000100:00080000:3.0:1787827471.562143:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827471.562145:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827471.562152:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827471.562160:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476707584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827471.562168:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827471.562170:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827471.562172:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827471.562175:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1839 00000100:00080000:3.0:1787827471.562176:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827471.562179:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827471.562183:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827471.562188:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476707456:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827471.562193:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476707712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827471.562201:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827471.562203:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827471.562204:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827471.562208:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1839 00000100:00080000:3.0:1787827471.562209:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827471.562211:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827471.562215:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827471.562224:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476707840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827471.562230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827471.562232:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827471.562233:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827471.562236:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1839 00000100:00080000:3.0:1787827471.562239:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827471.562241:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827471.562243:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:1.0:1787827471.562249:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827471.562253:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476707968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827471.562292:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827471/real 1787827471] req@ffff9a78a6775680 x1874671476707456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827471.562305:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476707456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827471.562314:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827471.562316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827471.562319:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476707456:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827471.562325:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707584:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827471.562338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827471.562346:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827471/real 1787827471] req@ffff9a78a6776d00 x1874671476707584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827471.562354:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476707584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827471.562359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827471.562362:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827471.562364:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707584:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827471.562384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707712:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827471.562394:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827471.562399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827471/real 1787827471] req@ffff9a78a67761c0 x1874671476707712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827471.562407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476707712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827471.562412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827471.562414:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827471.562416:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707712:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827471.562420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707840:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827471.562429:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827471.562435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827471/real 1787827471] req@ffff9a78a6774780 x1874671476707840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827471.562441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476707840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827471.562453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827471.562455:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827471.562457:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707840:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827471.562460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707968:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827471.562470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827471.562476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827471/real 1787827471] req@ffff9a78a67770c0 x1874671476707968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827471.562483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476707968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827471.562489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827471.562490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827471.562492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476707968:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827472.586000:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827473.610050:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827474.634113:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827475.657999:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827476.681959:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827476.681970:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827476.681975:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827476.681986:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1844 00000100:00080000:1.0:1787827476.681998:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827476.682006:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827476.682021:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827476.682054:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476708096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827476.682089:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827476.682094:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827476.682096:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827476.682100:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1844 00000100:00080000:1.0:1787827476.682104:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827476.682109:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827476.682122:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827476.682141:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476708224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827476.682141:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476708096:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827476.682157:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827476.682168:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827476.682169:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827476.682173:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1844 00000100:00080000:1.0:1787827476.682177:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827476.682181:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827476.682186:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827476.682196:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476708352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827476.682204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827476.682206:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827476.682207:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827476.682209:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1844 00000100:00080000:1.0:1787827476.682212:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787827476.682217:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827476.682219:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827476.682225:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827476.682235:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476708480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827476.682241:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827476.682244:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827476.682245:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827476.682248:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1844 00000100:00080000:1.0:1787827476.682249:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827476.682251:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827476.682254:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827476.682261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476708608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827476.682284:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827476/real 1787827476] req@ffff9a7890b1f480 x1874671476708096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827476.682301:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476708096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827476.682313:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827476.682318:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827476.682323:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476708096:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827476.682353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708224:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827476.682379:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827476.682399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827476/real 1787827476] req@ffff9a7890b1f840 x1874671476708224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827476.682413:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476708224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827476.682423:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827476.682427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827476.682430:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708224:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827476.682437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708352:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827476.682457:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827476.682469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827476/real 1787827476] req@ffff9a7890b1da40 x1874671476708352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827476.682483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476708352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827476.682492:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827476.682496:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827476.682500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708352:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827476.682507:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708480:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827476.682529:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827476.682543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827476/real 1787827476] req@ffff9a7890b1e940 x1874671476708480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827476.682556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476708480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827476.682568:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827476.682572:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827476.682575:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708480:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827476.682581:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708608:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827476.682605:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827476.682619:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827476/real 1787827476] req@ffff9a7890b1f0c0 x1874671476708608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827476.682633:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476708608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827501 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827476.682643:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827476.682647:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827476.682651:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708608:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827477.706058:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827478.729972:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827479.754003:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827480.778005:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827481.802013:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827481.802024:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827481.802030:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827481.802041:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1849 00000100:00080000:2.0:1787827481.802054:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827481.802061:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827481.802062:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827481.802112:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476708736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827481.802143:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827481.802147:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827481.802148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827481.802152:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1849 00000100:00080000:2.0:1787827481.802155:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827481.802158:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827481.802168:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827481.802178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476708864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827481.802189:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476708736:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827481.802189:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827481.802195:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827481.802199:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827481.802203:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1849 00000100:00080000:2.0:1787827481.802205:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827481.802208:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827481.802211:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827481.802221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476708992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827481.802228:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827481.802231:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827481.802233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827481.802237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1849 00000100:00080000:2.0:1787827481.802278:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827481.802282:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827481.802288:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827481.802295:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827481.802302:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476709120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827481.802312:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827481.802315:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827481.802332:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827481.802336:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1849 00000100:00080000:2.0:1787827481.802343:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827481.802347:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827481.802350:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827481.802360:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476709248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827481.802414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827481/real 1787827481] req@ffff9a78841da580 x1874671476708736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827481.802433:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476708736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827481.802454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827481.802457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827481.802461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476708736:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827481.802487:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708864:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827481.802508:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827481.802520:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827481/real 1787827481] req@ffff9a78841d8000 x1874671476708864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827481.802530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476708864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827481.802536:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827481.802539:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827481.802541:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708864:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827481.802546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708992:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827481.802558:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827481.802564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827481/real 1787827481] req@ffff9a78841db480 x1874671476708992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827481.802574:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476708992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827481.802584:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827481.802588:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827481.802592:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476708992:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827481.802616:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709120:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827481.802644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827481.802658:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827481/real 1787827481] req@ffff9a78841d9e00 x1874671476709120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827481.802692:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476709120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827481.802702:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827481.802704:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827481.802708:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709120:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827481.802713:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709248:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827481.802750:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827481.802775:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827481/real 1787827481] req@ffff9a78841d83c0 x1874671476709248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827481.802789:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476709248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827506 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827481.802801:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827481.802805:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827481.802809:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709248:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827482.826206:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827483.850104:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827484.874050:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827485.897938:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827486.921917:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827486.921944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827486.921953:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827486.921958:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827486.921969:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1854 00000100:00080000:2.0:1787827486.921980:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827486.921987:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827486.922031:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476709376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827486.922056:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827486.922060:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827486.922061:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827486.922064:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1854 00000100:00080000:2.0:1787827486.922067:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827486.922069:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827486.922080:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827486.922089:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476709504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827486.922099:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827486.922104:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827486.922105:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476709376:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827486.922106:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827486.922120:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1854 00000100:00080000:2.0:1787827486.922124:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827486.922126:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827486.922130:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827486.922142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476709632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827486.922149:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827486.922152:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827486.922154:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827486.922156:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1854 00000100:00080000:2.0:1787827486.922172:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827486.922175:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827486.922180:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827486.922187:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827486.922191:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476709760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827486.922200:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827486.922203:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827486.922206:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827486.922212:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1854 00000100:00080000:2.0:1787827486.922214:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827486.922217:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827486.922221:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827486.922233:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476709888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827486.922250:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827486/real 1787827486] req@ffff9a78841dbc00 x1874671476709376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827486.922276:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476709376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827486.922295:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827486.922299:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827486.922303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476709376:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827486.922325:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709504:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827486.922355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827486.922365:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827486/real 1787827486] req@ffff9a78841da1c0 x1874671476709504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827486.922373:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476709504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827486.922379:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827486.922381:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827486.922384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709504:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827486.922389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709632:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827486.922401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827486.922407:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827486/real 1787827486] req@ffff9a78841db0c0 x1874671476709632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827486.922414:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476709632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827486.922419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827486.922421:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827486.922424:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709632:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827486.922429:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709760:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827486.922439:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827486.922445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827486/real 1787827486] req@ffff9a78841d9a40 x1874671476709760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827486.922452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476709760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827486.922458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827486.922460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827486.922462:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709760:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827486.922466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709888:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827486.922476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827486.922481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827486/real 1787827486] req@ffff9a78841d8780 x1874671476709888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827486.922489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476709888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827511 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827486.922494:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827486.922496:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827486.922498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476709888:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827487.945982:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827488.969989:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827489.993917:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827491.017945:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827492.041937:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827492.041948:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827492.041953:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827492.041964:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1859 00000100:00080000:1.0:1787827492.041976:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827492.041984:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827492.042029:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476710016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827492.042055:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827492.042059:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827492.042060:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827492.042065:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1859 00010000:00010000:0.0:1787827492.042068:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827492.042069:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827492.042078:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827492.042090:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827492.042095:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476710016:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827492.042108:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476710144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827492.042121:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827492.042123:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827492.042125:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827492.042129:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1859 00000100:00080000:1.0:1787827492.042132:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827492.042135:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827492.042140:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827492.042150:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476710272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827492.042162:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827492.042166:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:2.0:1787827492.042167:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827492.042169:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827492.042174:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1859 00000100:00080000:1.0:1787827492.042179:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827492.042181:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827492.042186:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827492.042195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476710400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827492.042201:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827492.042204:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827492.042205:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827492.042209:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1859 00000100:00080000:1.0:1787827492.042212:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827492.042215:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827492.042222:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9a7890b1fc00 x1874671476710016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787827492.042226:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827492.042239:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476710528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827492.042244:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476710016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827492.042257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827492.042261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827492.042264:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476710016:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827492.042287:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710144:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827492.042303:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827492.042311:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9a7890b1c000 x1874671476710144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827492.042319:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476710144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827492.042324:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827492.042326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827492.042328:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710144:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827492.042332:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710272:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827492.042342:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827492.042348:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9a7890b1f0c0 x1874671476710272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827492.042356:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476710272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827492.042361:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827492.042362:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827492.042364:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710272:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827492.042367:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710400:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827492.042377:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827492.042383:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9a7890b1e940 x1874671476710400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827492.042395:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476710400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827492.042402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827492.042404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827492.042406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710400:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827492.042410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710528:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827492.042419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827492.042425:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9a7890b1da40 x1874671476710528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827492.042432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476710528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827492.042439:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827492.042441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827492.042443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710528:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827493.066053:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827494.090101:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827495.114018:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827496.138119:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827497.161933:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827497.161945:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827497.161951:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827497.161966:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1864 00000100:00080000:1.0:1787827497.161982:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827497.161992:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827497.161996:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827497.162040:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476710656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827497.162064:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827497.162067:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827497.162069:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827497.162073:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1864 00000100:00080000:1.0:1787827497.162076:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827497.162080:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827497.162091:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827497.162096:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476710656:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827497.162103:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476710784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827497.162111:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827497.162114:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827497.162115:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827497.162118:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1864 00000100:00080000:1.0:1787827497.162120:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827497.162122:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827497.162126:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827497.162144:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476710912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827497.162151:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827497.162155:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827497.162156:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827497.162159:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1864 00000100:00080000:1.0:1787827497.162168:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827497.162171:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827497.162175:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787827497.162177:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827497.162186:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476711040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827497.162195:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827497.162197:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827497.162199:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827497.162201:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1864 00000100:00080000:1.0:1787827497.162203:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827497.162206:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827497.162208:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827497.162217:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476711168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827497.162239:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff9a7890b1f840 x1874671476710656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.162256:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476710656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.162270:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.162273:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827497.162276:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476710656:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827497.162298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710784:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827497.162314:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827497.162323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff9a7890b1f480 x1874671476710784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.162331:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476710784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.162336:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.162339:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827497.162341:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710784:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827497.162345:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710912:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827497.162355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827497.162360:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff9a7890b1e580 x1874671476710912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.162367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476710912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.162373:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.162376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827497.162380:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476710912:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827497.162387:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711040:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827497.162407:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827497.162420:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff9a7890b1de00 x1874671476711040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.162433:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476711040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.162441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.162443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827497.162445:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711040:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827497.162449:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711168:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827497.162465:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827497.162472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827497/real 1787827497] req@ffff9a7890b1c3c0 x1874671476711168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827497.162479:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476711168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827497.162486:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827497.162488:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827497.162490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711168:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827498.186060:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827499.210013:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827500.233979:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827501.257960:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827502.281981:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827502.281994:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827502.281999:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827502.282010:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1869 00000100:00080000:2.0:1787827502.282021:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827502.282040:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827502.282054:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827502.282080:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476711296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827502.282123:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827502.282127:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827502.282129:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827502.282133:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1869 00000100:00080000:2.0:1787827502.282138:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827502.282144:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827502.282155:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827502.282174:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476711296:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827502.282182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476711424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827502.282197:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827502.282201:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827502.282203:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827502.282209:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1869 00000100:00080000:2.0:1787827502.282242:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827502.282247:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827502.282252:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787827502.282256:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827502.282274:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476711552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827502.282294:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827502.282297:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827502.282300:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827502.282307:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9a78841d8780 x1874671476711296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827502.282308:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1869 00000100:00080000:2.0:1787827502.282315:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827502.282322:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827502.282332:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827502.282339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476711296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827502.282348:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476711680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827502.282365:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827502.282370:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827502.282371:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827502.282379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476711296:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827502.282380:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827502.282387:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827502.282393:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1869 00000100:00080000:2.0:1787827502.282397:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827502.282402:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827502.282409:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827502.282415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711424:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827502.282424:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476711808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827502.282440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827502.282465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9a78841d9a40 x1874671476711424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827502.282475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476711424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827502.282481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827502.282484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827502.282486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711424:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827502.282490:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711552:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827502.282502:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827502.282509:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9a78841db0c0 x1874671476711552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827502.282517:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476711552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827502.282523:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827502.282525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827502.282527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711552:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827502.282541:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711680:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827502.282553:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827502.282559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9a78841da1c0 x1874671476711680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827502.282567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476711680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827502.282574:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827502.282576:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827502.282578:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711680:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827502.282590:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711808:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827502.282601:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827502.282608:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827502/real 1787827502] req@ffff9a78841dbc00 x1874671476711808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827502.282615:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476711808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827502.282622:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827502.282624:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827502.282626:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476711808:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827503.305958:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827504.330011:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827505.353960:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827506.377989:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827507.401936:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.401948:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827507.401952:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.401961:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1874 00000100:00080000:1.0:1787827507.401971:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827507.401972:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827507.401978:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827507.402010:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476711936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827507.402038:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.402043:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827507.402045:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.402049:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1874 00000100:00080000:1.0:1787827507.402053:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827507.402058:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827507.402069:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827507.402080:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476712064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827507.402085:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476711936:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827507.402090:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.402095:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827507.402097:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.402100:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1874 00000100:00080000:1.0:1787827507.402103:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827507.402105:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827507.402109:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827507.402119:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476712192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827507.402130:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.402132:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827507.402133:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.402136:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1874 00000100:00080000:1.0:1787827507.402138:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827507.402140:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827507.402143:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787827507.402148:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827507.402153:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476712320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827507.402161:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827507.402164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827507.402165:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827507.402168:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1874 00000100:00080000:1.0:1787827507.402170:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827507.402173:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827507.402176:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827507.402184:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476712448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827507.402196:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9a7890b1d680 x1874671476711936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.402226:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476711936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.402235:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.402238:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827507.402241:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476711936:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827507.402261:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712064:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827507.402279:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827507.402289:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9a7890b1cf00 x1874671476712064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.402297:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476712064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.402302:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.402304:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827507.402306:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712064:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827507.402310:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712192:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827507.402320:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827507.402325:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9a7890b1c3c0 x1874671476712192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.402336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476712192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.402351:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.402354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827507.402358:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712192:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827507.402365:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712320:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827507.402385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827507.402394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9a7890b1de00 x1874671476712320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.402402:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476712320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.402408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.402410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827507.402412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712320:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827507.402428:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712448:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827507.402445:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827507.402456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827507/real 1787827507] req@ffff9a7890b1e580 x1874671476712448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827507.402464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476712448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827507.402470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827507.402472:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827507.402475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712448:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827508.425972:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827509.449972:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827510.474020:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827511.498015:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827512.521942:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827512.521951:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827512.521956:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827512.521967:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1880 00000100:00080000:3.0:1787827512.521978:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827512.521987:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827512.522024:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476712576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827512.522052:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827512.522056:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827512.522059:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827512.522063:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1880 00000100:00080000:3.0:1787827512.522067:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827512.522072:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827512.522082:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827512.522098:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476712704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827512.522100:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476712576:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827512.522117:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827512.522121:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827512.522123:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827512.522128:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1880 00000100:00080000:3.0:1787827512.522132:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827512.522136:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827512.522142:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827512.522155:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476712832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827512.522165:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827512.522180:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827512.522183:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827512.522185:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827512.522188:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1880 00000100:00080000:3.0:1787827512.522192:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827512.522195:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827512.522200:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827512.522211:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827512/real 1787827512] req@ffff9a78a6776940 x1874671476712576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827512.522212:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476712960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827512.522244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827512.522248:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827512.522249:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827512.522253:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476712576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827512.522256:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1880 00000100:00080000:3.0:1787827512.522262:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827512.522268:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827512.522274:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787827512.522276:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827512.522282:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827512.522284:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476713088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827512.522288:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476712576:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827512.522315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712704:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827512.522335:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827512.522347:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827512/real 1787827512] req@ffff9a78a6776580 x1874671476712704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827512.522357:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476712704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827512.522364:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827512.522366:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827512.522370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712704:192.168.203.140@tcp:38:kworker.0 00010000:00010000:0.0:1787827512.522371:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827512.522377:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712832:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827512.522389:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827512.522396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827512/real 1787827512] req@ffff9a78a6774000 x1874671476712832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827512.522403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476712832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827512.522408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827512.522410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827512.522411:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712832:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827512.522415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712960:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827512.522424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827512.522430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827512/real 1787827512] req@ffff9a78a6775a40 x1874671476712960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827512.522437:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476712960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827512.522443:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827512.522444:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827512.522446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476712960:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827512.522449:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713088:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827512.522458:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827512.522463:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827512/real 1787827512] req@ffff9a78a6777480 x1874671476713088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827512.522470:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476713088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827512.522476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827512.522478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827512.522480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713088:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827513.545988:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827514.569976:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827515.594071:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827516.618059:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827517.641953:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827517.641961:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827517.641966:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827517.641975:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1885 00010000:00010000:0.0:1787827517.641978:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827517.641985:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827517.642000:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827517.642031:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476713216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827517.642053:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827517.642055:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827517.642057:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827517.642059:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1885 00000100:00080000:1.0:1787827517.642062:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827517.642064:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827517.642071:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827517.642078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476713344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827517.642086:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827517.642089:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827517.642090:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827517.642093:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1885 00000100:00080000:1.0:1787827517.642095:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827517.642097:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827517.642099:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827517.642107:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476713472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827517.642115:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827517.642118:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827517.642119:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827517.642122:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1885 00000100:00080000:1.0:1787827517.642124:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827517.642127:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827517.642129:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827517.642135:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476713600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827517.642140:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827517.642141:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827517.642142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827517.642145:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1885 00000100:00080000:1.0:1787827517.642149:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827517.642151:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827517.642154:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827517.642160:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476713728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827517.642179:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476713216:192.168.203.140@tcp:250:kworker.0 00000800:00000100:2.0:1787827517.642308:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827517.642408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827517/real 1787827517] req@ffff9a7890b1f480 x1874671476713216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827517.642440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476713216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827517.642459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827517.642467:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827517.642474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476713216:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827517.642488:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713344:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827517.642536:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827517.642553:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827517/real 1787827517] req@ffff9a7890b1f840 x1874671476713344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827517.642571:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476713344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827517.642581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827517.642626:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827517.642632:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713344:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827517.642646:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713472:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827517.642689:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827517.642703:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827517/real 1787827517] req@ffff9a7890b1da40 x1874671476713472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827517.642717:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476713472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827517.642726:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827517.642730:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827517.642734:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713472:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827517.642760:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713600:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827517.642784:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827517.642796:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827517/real 1787827517] req@ffff9a7890b1e940 x1874671476713600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827517.642818:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476713600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827517.642854:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827517.642860:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827517.642865:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713600:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827517.642967:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713728:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827517.643000:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827517.643020:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827517/real 1787827517] req@ffff9a7890b1f0c0 x1874671476713728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827517.643043:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476713728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827517.643059:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827517.643064:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827517.643068:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713728:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827518.666043:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827519.689985:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827520.713972:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827521.738022:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827522.761973:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827522.761986:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827522.761992:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827522.762004:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1890 00000100:00080000:3.0:1787827522.762018:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827522.762026:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827522.762080:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476713856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827522.762111:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827522.762115:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827522.762118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827522.762122:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1890 00000100:00080000:3.0:1787827522.762126:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827522.762136:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827522.762148:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827522.762161:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476713984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827522.762168:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476713856:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827522.762177:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827522.762185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827522.762187:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827522.762193:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1890 00000100:00080000:3.0:1787827522.762198:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827522.762201:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827522.762207:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827522.762221:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476714112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827522.762231:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827522.762235:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827522.762237:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827522.762242:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1890 00000100:00080000:3.0:1787827522.762245:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827522.762250:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827522.762255:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827522.762266:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476714240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827522.762276:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827522.762279:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827522.762282:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827522.762288:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1890 00000100:00080000:3.0:1787827522.762292:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827522.762297:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827522.762305:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827522.762306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713984:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827522.762326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476714368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827522.762339:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714112:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827522.762365:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714240:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827522.762393:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714368:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827522.762459:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827523.786014:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827524.810070:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827525.834055:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827525.834136:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787827525.834176:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827525.834193:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827525.834250:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827525.834261:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827525.834273:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827525.834286:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787827525.834302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827522/real 1787827525] req@ffff9a78a6774b40 x1874671476713984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827525.834330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476713984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827525.834344:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827525.834349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827525.834354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476713984:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827525.834362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827522/real 1787827525] req@ffff9a78a6777480 x1874671476714112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827525.834379:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476714112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827525.834388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827525.834407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827525.834411:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714112:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827525.834418:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827522/real 1787827525] req@ffff9a78a6775a40 x1874671476714240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827525.834431:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476714240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827525.834442:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827525.834445:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827525.834448:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714240:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827525.834455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827522/real 1787827525] req@ffff9a78a67752c0 x1874671476713856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827525.834467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476713856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827525.834475:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827525.834478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827525.834482:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476713856:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827525.834518:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827522/real 1787827525] req@ffff9a78a6774000 x1874671476714368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827525.834548:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476714368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827547 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827525.834558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827525.834562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827525.834566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714368:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827526.857958:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827527.881915:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827527.881924:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827527.881929:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827527.881940:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1895 00000100:00080000:3.0:1787827527.881951:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827527.881958:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827527.881989:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476714496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827527.882017:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827527.882025:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827527.882028:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827527.882032:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1895 00000100:00080000:3.0:1787827527.882036:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827527.882041:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827527.882051:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827527.882066:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476714496:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827527.882067:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476714624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827527.882084:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827527.882093:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827527.882095:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827527.882099:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1895 00000100:00080000:3.0:1787827527.882103:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827527.882108:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827527.882114:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827527.882127:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476714752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827527.882137:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827527.882139:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827527.882144:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827527.882146:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827527.882152:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1895 00000100:00080000:3.0:1787827527.882157:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827527.882166:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827527.882172:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827527.882189:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827527/real 1787827527] req@ffff9a78a6776580 x1874671476714496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827527.882190:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476714880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827527.882215:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827527.882219:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827527.882221:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827527.882227:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1895 00000100:00100000:2.0:1787827527.882230:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476714496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827527.882232:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827527.882241:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827527.882251:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787827527.882254:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827527.882262:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827527.882265:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476715008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827527.882272:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476714496:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827527.882304:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714624:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827527.882339:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827527.882355:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827527/real 1787827527] req@ffff9a78a6776940 x1874671476714624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827527.882370:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476714624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827527.882384:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827527.882387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827527.882390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714624:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827527.882396:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714752:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827527.882418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:0.0:1787827527.882419:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827527.882432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827527/real 1787827527] req@ffff9a78a67770c0 x1874671476714752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827527.882446:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476714752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827527.882456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827527.882458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827527.882461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714752:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827527.882466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714880:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827527.882485:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827527.882495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827527/real 1787827527] req@ffff9a78a6774780 x1874671476714880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827527.882509:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476714880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827527.882536:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827527.882540:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827527.882545:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476714880:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827527.882551:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715008:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827527.882572:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827527.882584:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827527/real 1787827527] req@ffff9a78a67761c0 x1874671476715008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827527.882595:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476715008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827552 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827527.882605:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827527.882607:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827527.882610:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715008:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827528.906051:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827529.930179:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827530.953998:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827531.977978:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827533.001899:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827533.001910:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827533.001914:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827533.001925:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1900 00000100:00080000:3.0:1787827533.001937:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827533.001944:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827533.001975:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476715136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827533.002016:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827533.002018:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827533.002020:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827533.002022:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1900 00000100:00080000:3.0:1787827533.002025:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827533.002027:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827533.002034:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827533.002042:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476715264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827533.002049:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827533.002052:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827533.002053:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827533.002057:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1900 00000100:00080000:3.0:1787827533.002059:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827533.002062:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827533.002066:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827533.002072:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476715392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827533.002084:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827533.002088:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827533.002089:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827533.002096:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1900 00000100:00080000:3.0:1787827533.002100:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827533.002102:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827533.002105:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476715136:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787827533.002105:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827533.002116:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476715520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827533.002123:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827533.002125:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827533.002126:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827533.002130:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1900 00000100:00080000:3.0:1787827533.002133:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827533.002137:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827533.002140:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827533.002148:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476715648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827533.002191:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:1.0:1787827533.002242:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827533.002253:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9a78a6776d00 x1874671476715136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827533.002269:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476715136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827533.002280:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827533.002283:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827533.002287:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476715136:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827533.002304:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715264:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827533.002320:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827533.002329:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9a78a6775680 x1874671476715264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827533.002337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476715264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827533.002363:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827533.002367:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827533.002372:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715264:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827533.002428:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715392:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827533.002463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827533.002507:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9a78a6777840 x1874671476715392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827533.002536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476715392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827533.002542:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827533.002544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827533.002546:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715392:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827533.002551:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715520:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827533.002566:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827533.002573:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9a78a6774f00 x1874671476715520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827533.002581:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476715520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827533.002589:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827533.002591:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827533.002593:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715520:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827533.002598:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715648:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827533.002608:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827533.002613:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827533/real 1787827533] req@ffff9a78a67743c0 x1874671476715648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827533.002621:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476715648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827533.002627:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827533.002629:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827533.002631:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715648:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827534.026037:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827535.050005:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827536.074044:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827537.098035:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827538.121942:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827538.121953:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827538.121958:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827538.121971:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1905 00010000:00010000:0.0:1787827538.121981:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827538.121986:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827538.121997:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827538.122045:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476715776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827538.122073:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827538.122077:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827538.122079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827538.122081:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1905 00000100:00080000:2.0:1787827538.122085:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827538.122088:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827538.122098:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827538.122107:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476715904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827538.122115:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827538.122117:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827538.122119:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827538.122123:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1905 00000100:00080000:2.0:1787827538.122124:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827538.122127:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827538.122130:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827538.122136:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476716032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827538.122144:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476715776:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827538.122145:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827538.122152:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827538.122155:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827538.122159:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1905 00000100:00080000:2.0:1787827538.122161:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827538.122163:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827538.122168:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827538.122175:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476716160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827538.122182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827538.122184:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827538.122186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827538.122189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1905 00000100:00080000:2.0:1787827538.122191:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827538.122193:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827538.122196:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827538.122202:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476716288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827538.122239:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827538.122304:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff9a78841d83c0 x1874671476715776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827538.122326:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476715776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827538.122338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827538.122343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827538.122349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476715776:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827538.122357:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715904:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827538.122383:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827538.122397:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff9a78841d9e00 x1874671476715904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827538.122411:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476715904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827538.122420:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827538.122424:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827538.122428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476715904:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827538.122473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716032:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827538.122496:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827538.122509:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff9a78841db480 x1874671476716032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827538.122532:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476716032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827538.122541:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827538.122545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827538.122549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716032:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827538.122556:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716160:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827538.122578:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827538.122592:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff9a78841d8000 x1874671476716160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827538.122605:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476716160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827538.122617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827538.122621:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827538.122625:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716160:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827538.122631:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716288:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827538.122652:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827538.122666:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827538/real 1787827538] req@ffff9a78841da580 x1874671476716288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827538.122679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476716288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827538.122698:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827538.122701:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827538.122705:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716288:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827539.146122:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827540.170082:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827541.194061:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827542.217953:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827543.241957:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.241966:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827543.241969:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827543.241989:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1910 00000100:00080000:2.0:1787827543.241999:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787827543.242005:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827543.242005:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827543.242039:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476716416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827543.242059:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.242062:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827543.242063:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827543.242066:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1910 00000100:00080000:2.0:1787827543.242068:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827543.242071:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827543.242079:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827543.242088:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476716544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827543.242097:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476716416:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827543.242098:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.242107:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827543.242108:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827543.242112:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1910 00000100:00080000:2.0:1787827543.242114:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827543.242116:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827543.242120:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827543.242128:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476716672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827543.242134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.242137:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827543.242138:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827543.242140:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1910 00000100:00080000:2.0:1787827543.242150:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827543.242152:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827543.242155:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827543.242159:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827543.242168:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476716800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827543.242180:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827543.242182:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827543.242184:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827543.242187:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1910 00000100:00080000:2.0:1787827543.242190:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827543.242193:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827543.242196:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827543.242211:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827543] req@ffff9a78841d8b40 x1874671476716416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827543.242218:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476716928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827543.242232:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476716416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827543.242242:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827543.242246:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827543.242248:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476716416:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827543.242279:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716544:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827543.242294:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827543.242302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827543] req@ffff9a78841db840 x1874671476716544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827543.242310:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476716544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827543.242320:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827543.242322:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827543.242325:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716544:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827543.242328:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716672:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827543.242338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827543.242343:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827543] req@ffff9a78841d92c0 x1874671476716672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827543.242350:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476716672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827543.242356:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827543.242357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827543.242359:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716672:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827543.242363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716800:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827543.242372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827543.242377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827543] req@ffff9a78841d9680 x1874671476716800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827543.242384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476716800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827543.242392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827543.242395:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827543.242399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716800:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827543.242416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716928:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827543.242444:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827543.242456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827543/real 1787827543] req@ffff9a78841da940 x1874671476716928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827543.242467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476716928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827543.242474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827543.242476:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827543.242479:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476716928:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827544.265959:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827545.289979:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827546.313986:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827547.337991:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827548.361904:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827548.361913:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827548.361916:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827548.361925:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1915 00000100:00080000:2.0:1787827548.361934:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827548.361939:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827548.361972:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827548.361972:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476717056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827548.361997:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827548.362000:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827548.362001:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827548.362004:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1915 00000100:00080000:2.0:1787827548.362006:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827548.362009:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827548.362016:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827548.362026:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476717184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827548.362034:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827548.362037:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827548.362038:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827548.362043:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1915 00000100:00100000:1.0:1787827548.362046:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476717056:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827548.362046:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827548.362052:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827548.362058:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827548.362068:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476717312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827548.362075:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827548.362078:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827548.362079:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827548.362081:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1915 00000100:00080000:2.0:1787827548.362096:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827548.362099:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827548.362103:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827548.362107:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827548.362115:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476717440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827548.362123:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827548.362126:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827548.362128:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827548.362130:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1915 00000100:00080000:2.0:1787827548.362133:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827548.362135:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827548.362139:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827548.362150:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff9a78841da940 x1874671476717056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827548.362150:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476717568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827548.362172:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476717056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827548.362182:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827548.362185:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827548.362188:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476717056:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827548.362205:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717184:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827548.362225:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827548.362239:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff9a78841d9680 x1874671476717184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827548.362252:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476717184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827548.362261:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827548.362264:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827548.362268:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717184:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827548.362274:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717312:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827548.362305:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827548.362316:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff9a78841d92c0 x1874671476717312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827548.362325:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476717312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827548.362331:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827548.362333:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827548.362336:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717312:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827548.362340:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717440:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827548.362352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827548.362359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff9a78841db840 x1874671476717440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827548.362408:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476717440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827548.362416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827548.362418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827548.362420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717440:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827548.362424:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717568:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827548.362438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827548.362445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827548/real 1787827548] req@ffff9a78841d8b40 x1874671476717568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827548.362452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476717568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827548.362458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827548.362460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827548.362462:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717568:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827549.386009:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827550.409938:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827551.434043:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827552.457956:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827553.481930:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827553.481935:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827553.481941:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827553.481946:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827553.481954:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1920 00000100:00080000:1.0:1787827553.481962:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827553.481967:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827553.481998:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476717696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827553.482022:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827553.482025:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827553.482026:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827553.482029:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1920 00000100:00080000:1.0:1787827553.482032:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827553.482035:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827553.482043:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827553.482051:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476717824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827553.482057:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476717696:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827553.482062:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827553.482065:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827553.482067:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827553.482070:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1920 00000100:00080000:1.0:1787827553.482072:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827553.482074:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827553.482078:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827553.482086:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476717952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827553.482093:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827553.482095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827553.482097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827553.482099:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1920 00000100:00080000:1.0:1787827553.482104:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827553.482106:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827553.482110:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787827553.482115:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827553.482122:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476718080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827553.482130:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827553.482132:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827553.482133:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827553.482136:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1920 00000100:00080000:1.0:1787827553.482138:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827553.482141:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827553.482144:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827553.482152:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476718208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827553.482158:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827553/real 1787827553] req@ffff9a7890b1c000 x1874671476717696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827553.482175:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476717696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827553.482201:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827553.482206:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827553.482210:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476717696:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827553.482237:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717824:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827553.482264:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827553.482293:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827553/real 1787827553] req@ffff9a7890b1fc00 x1874671476717824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827553.482305:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476717824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827553.482319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827553.482322:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827553.482325:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717824:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827553.482330:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717952:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827553.482384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827553.482392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827553/real 1787827553] req@ffff9a7890b1f0c0 x1874671476717952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827553.482400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476717952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827553.482405:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827553.482407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827553.482410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476717952:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827553.482414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718080:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827553.482443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827553.482452:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827553/real 1787827553] req@ffff9a7890b1e940 x1874671476718080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827553.482463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476718080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827553.482471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827553.482474:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827553.482477:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718080:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827553.482483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718208:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827553.482516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827553.482526:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827553/real 1787827553] req@ffff9a7890b1da40 x1874671476718208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827553.482537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476718208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827553.482547:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827553.482568:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827553.482571:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718208:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827554.505948:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827555.529960:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827556.553927:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827557.577972:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827558.601927:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827558.601935:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827558.601939:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827558.601947:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1926 00000100:00080000:3.0:1787827558.601957:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827558.601963:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827558.601993:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476718336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827558.602020:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827558.602024:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827558.602027:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827558.602031:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1926 00000100:00080000:3.0:1787827558.602035:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827558.602039:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827558.602048:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827558.602063:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476718464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827558.602074:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476718336:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827558.602078:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827558.602083:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827558.602085:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827558.602090:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1926 00000100:00080000:3.0:1787827558.602094:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827558.602099:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827558.602105:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827558.602119:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476718592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827558.602131:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827558.602132:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827558.602138:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827558.602141:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827558.602147:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1926 00000100:00080000:3.0:1787827558.602151:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827558.602155:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827558.602161:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827558.602173:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476718720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827558.602183:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9a78a6777c00 x1874671476718336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.602186:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827558.602196:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827558.602201:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827558.602209:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1926 00000100:00100000:2.0:1787827558.602211:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476718336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.602214:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827558.602225:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:2.0:1787827558.602227:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827558.602232:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:3.0:1787827558.602232:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827558.602240:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476718336:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827558.602251:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476718848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827558.602261:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718464:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827558.602282:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827558.602314:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9a78a6775e00 x1874671476718464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827558.602331:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476718464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827558.602341:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00010000:00010000:0.0:1787827558.602347:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827558.602347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827558.602354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718464:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827558.602362:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718592:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827558.602378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827558.602386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9a78a67743c0 x1874671476718592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827558.602394:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476718592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827558.602399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827558.602401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827558.602403:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718592:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827558.602406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718720:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827558.602416:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827558.602422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9a78a6774f00 x1874671476718720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827558.602429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476718720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827558.602435:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827558.602438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827558.602439:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718720:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827558.602452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718848:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827558.602461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827558.602467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9a78a6777840 x1874671476718848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827558.602474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476718848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827558.602480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827558.602482:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827558.602484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476718848:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827559.626064:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827560.650087:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827561.674003:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827562.698132:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827563.722015:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827563.722033:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827563.722041:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827563.722058:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1931 00010000:00010000:0.0:1787827563.722065:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827563.722077:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827563.722089:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827563.722145:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476718976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827563.722204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827563.722210:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827563.722213:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827563.722218:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1931 00000100:00080000:2.0:1787827563.722222:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827563.722227:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827563.722241:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827563.722256:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476719104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827563.722295:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827563.722300:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827563.722304:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827563.722320:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1931 00000100:00080000:2.0:1787827563.722326:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827563.722330:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827563.722338:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827563.722349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476718976:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827563.722359:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476719232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827563.722375:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827563.722379:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827563.722381:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827563.722387:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1931 00000100:00080000:2.0:1787827563.722409:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827563.722414:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827563.722422:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827563.722439:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476719360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827563.722452:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827563.722473:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827563.722476:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827563.722481:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1931 00000100:00080000:2.0:1787827563.722486:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827563.722492:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827563.722500:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:1.0:1787827563.722506:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827563.722549:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476719488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827563.722624:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9a78841da580 x1874671476718976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827563.722651:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476718976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827563.722668:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827563.722673:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827563.722680:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476718976:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827563.722701:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719104:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827563.722731:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827563.722747:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9a78841d8000 x1874671476719104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827563.722762:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476719104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827563.722771:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827563.722775:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827563.722779:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719104:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827563.722830:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719232:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827563.722923:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827563.722936:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9a78841db480 x1874671476719232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827563.722956:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476719232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827563.722967:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827563.722972:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827563.722977:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719232:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827563.722986:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719360:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827563.723017:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827563.723034:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9a78841d9e00 x1874671476719360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827563.723046:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476719360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827563.723057:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827563.723060:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827563.723063:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719360:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827563.723069:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719488:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827563.723088:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827563.723097:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9a78841d83c0 x1874671476719488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827563.723108:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476719488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827563.723117:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827563.723120:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827563.723123:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719488:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827564.746009:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827565.770111:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827566.793985:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827567.817963:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827568.841970:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.841984:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827568.841990:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827568.841992:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827568.842005:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1936 00000100:00080000:2.0:1787827568.842020:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.842029:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827568.842081:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476719616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827568.842126:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.842130:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827568.842131:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827568.842134:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1936 00000100:00080000:2.0:1787827568.842138:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.842141:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827568.842150:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827568.842176:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476719744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827568.842185:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.842189:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827568.842191:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827568.842194:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1936 00000100:00080000:2.0:1787827568.842199:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.842202:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827568.842215:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827568.842244:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476719872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827568.842253:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476719616:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827568.842259:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.842276:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827568.842291:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827568.842299:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1936 00000100:00080000:2.0:1787827568.842302:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.842305:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827568.842312:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827568.842326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476720000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827568.842342:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.842345:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827568.842347:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827568.842353:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1936 00000100:00080000:2.0:1787827568.842369:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.842373:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827568.842377:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:1.0:1787827568.842386:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827568.842391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476720128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827568.842464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9a78841dbc00 x1874671476719616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827568.842485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476719616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827568.842493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827568.842497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827568.842500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476719616:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827568.842518:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719744:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827568.842535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827568.842544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9a78841da1c0 x1874671476719744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827568.842551:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476719744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827568.842557:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827568.842559:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827568.842561:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719744:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827568.842592:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719872:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827568.842602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827568.842608:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9a78841db0c0 x1874671476719872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827568.842616:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476719872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827568.842625:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827568.842627:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827568.842629:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476719872:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827568.842633:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720000:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827568.842642:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827568.842648:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9a78841d9a40 x1874671476720000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827568.842655:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476720000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827568.842674:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827568.842677:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827568.842681:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720000:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827568.842687:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720128:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827568.842706:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827568.842717:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9a78841d8780 x1874671476720128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827568.842730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476720128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827568.842738:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827568.842740:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827568.842743:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720128:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827569.865962:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827570.890097:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827571.914095:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827572.938050:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827573.961939:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.961950:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827573.961955:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.961966:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1941 00000100:00080000:2.0:1787827573.961988:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827573.961997:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827573.962005:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827573.962053:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476720256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827573.962080:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.962083:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827573.962084:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.962087:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1941 00000100:00080000:2.0:1787827573.962089:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827573.962092:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827573.962104:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827573.962123:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476720384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827573.962131:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.962134:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827573.962136:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.962139:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1941 00000100:00080000:2.0:1787827573.962143:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827573.962151:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476720256:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827573.962152:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827573.962164:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827573.962181:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476720512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827573.962205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.962210:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827573.962213:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.962218:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1941 00000100:00080000:2.0:1787827573.962256:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827573.962261:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827573.962268:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787827573.962270:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827573.962289:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476720640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827573.962298:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.962301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827573.962303:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.962307:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1941 00000100:00080000:2.0:1787827573.962312:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827573.962315:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827573.962320:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827573.962324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9a78841d8780 x1874671476720256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827573.962331:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476720768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827573.962363:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476720256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827573.962374:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827573.962380:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827573.962384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476720256:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827573.962391:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720384:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827573.962408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827573.962417:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9a78841d9a40 x1874671476720384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827573.962425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476720384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827573.962430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827573.962433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827573.962435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720384:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827573.962456:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720512:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827573.962468:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827573.962475:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9a78841db0c0 x1874671476720512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827573.962482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476720512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827573.962487:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827573.962489:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827573.962491:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720512:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827573.962494:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720640:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827573.962505:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827573.962510:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9a78841da1c0 x1874671476720640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827573.962517:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476720640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827573.962524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827573.962526:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827573.962528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720640:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827573.962531:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720768:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827573.962541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827573.962547:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9a78841dbc00 x1874671476720768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827573.962555:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476720768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827573.962561:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827573.962563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827573.962565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476720768:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827574.986054:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827576.010195:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827577.034068:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827578.058042:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827579.081973:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827579.081982:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827579.081986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827579.081994:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1946 00000100:00080000:1.0:1787827579.082018:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827579.082024:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827579.082031:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827579.082059:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476720896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827579.082100:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827579.082103:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827579.082105:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827579.082109:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1946 00000100:00080000:1.0:1787827579.082111:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827579.082114:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827579.082124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827579.082136:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476721024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827579.082174:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827579.082179:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827579.082181:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827579.082181:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476720896:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827579.082188:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1946 00000100:00080000:1.0:1787827579.082192:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827579.082195:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827579.082199:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827579.082224:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476721152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827579.082231:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827579.082233:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827579.082234:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827579.082237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1946 00000100:00080000:1.0:1787827579.082239:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827579.082241:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827579.082244:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827579.082252:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476721280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827579.082258:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827579.082260:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827579.082261:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827579.082264:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1946 00000100:00080000:1.0:1787827579.082267:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827579.082273:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787827579.082273:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827579.082280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827579.082287:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476721408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827579.082335:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff9a7890b1f840 x1874671476720896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827579.082356:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476720896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827579.082377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827579.082382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827579.082386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476720896:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827579.082410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721024:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827579.082437:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827579.082452:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff9a7890b1f480 x1874671476721024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827579.082466:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476721024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827579.082482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827579.082486:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827579.082489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721024:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827579.082496:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721152:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787827579.082528:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827579.082541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff9a7890b1e580 x1874671476721152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827579.082554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476721152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827579.082563:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827579.082566:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827579.082569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721152:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827579.082576:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721280:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827579.082608:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827579.082622:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff9a7890b1de00 x1874671476721280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827579.082634:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476721280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827579.082644:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827579.082648:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827579.082653:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721280:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787827579.082659:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721408:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787827579.082681:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827579.082695:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827579/real 1787827579] req@ffff9a7890b1c3c0 x1874671476721408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827579.082707:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476721408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827579.082716:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827579.082720:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827579.082723:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721408:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827580.106001:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827581.129966:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827582.154116:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827583.178096:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827584.202008:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827584.202019:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827584.202034:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827584.202045:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1951 00000100:00080000:2.0:1787827584.202056:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787827584.202065:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827584.202065:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827584.202107:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476721536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827584.202153:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827584.202157:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827584.202159:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827584.202163:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1951 00000100:00080000:2.0:1787827584.202167:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827584.202171:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827584.202182:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827584.202196:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476721664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827584.202205:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476721536:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827584.202222:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827584.202227:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827584.202228:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827584.202232:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1951 00000100:00080000:2.0:1787827584.202235:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827584.202238:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827584.202243:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827584.202254:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476721792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827584.202263:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827584.202266:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827584.202268:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827584.202271:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1951 00000100:00080000:2.0:1787827584.202300:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827584.202303:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827584.202308:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827584.202318:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827584.202321:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476721920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827584.202332:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827584.202336:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827584.202337:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827584.202341:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1951 00000100:00080000:2.0:1787827584.202345:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827584.202348:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827584.202352:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827584.202361:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476722048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827584.202386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff9a78841d83c0 x1874671476721536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.202411:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476721536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.202423:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.202428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827584.202433:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476721536:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827584.202470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721664:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827584.202499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827584.202516:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff9a78841d9e00 x1874671476721664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.202532:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476721664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.202544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.202548:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827584.202553:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721664:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827584.202582:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721792:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827584.202602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827584.202612:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff9a78841db480 x1874671476721792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.202622:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476721792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.202636:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.202638:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827584.202641:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721792:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827584.202645:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721920:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827584.202658:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827584.202665:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff9a78841d8000 x1874671476721920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.202674:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476721920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.202682:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.202684:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827584.202687:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476721920:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827584.202692:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722048:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827584.202704:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827584.202711:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827584/real 1787827584] req@ffff9a78841da580 x1874671476722048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827584.202719:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476722048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827584.202738:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827584.202740:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827584.202742:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722048:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827585.226020:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827586.250040:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827587.274050:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827588.298065:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827589.321940:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827589.321949:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827589.321953:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827589.321962:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1956 00000100:00080000:2.0:1787827589.321974:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827589.321976:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827589.321998:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827589.322035:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476722176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827589.322075:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827589.322078:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827589.322080:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827589.322083:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1956 00000100:00080000:2.0:1787827589.322099:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827589.322105:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827589.322125:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827589.322141:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476722304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827589.322142:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476722176:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827589.322158:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827589.322162:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827589.322165:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827589.322170:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1956 00000100:00080000:2.0:1787827589.322175:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827589.322187:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827589.322191:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827589.322201:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476722432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827589.322213:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827589.322218:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827589.322219:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827589.322223:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1956 00000100:00080000:2.0:1787827589.322227:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827589.322230:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827589.322235:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827589.322262:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476722560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827589.322270:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827589.322274:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:1.0:1787827589.322275:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722304:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787827589.322277:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827589.322286:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1956 00000100:00080000:2.0:1787827589.322292:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827589.322297:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827589.322303:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827589.322307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722432:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787827589.322336:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476722688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827589.322339:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722560:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827589.322366:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722688:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827590.346372:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827591.370036:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827592.394135:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827592.394196:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787827592.394249:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827592.394265:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827592.394311:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827592.394323:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827592.394333:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827592.394342:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827592.394720:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827592] req@ffff9a78841da940 x1874671476722688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827592.394771:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476722688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827592.394787:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827592.394792:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827592.394802:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722688:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827592.394811:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827592] req@ffff9a78841db840 x1874671476722304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827592.394825:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476722304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827592.394855:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827592.394861:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827592.394866:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722304:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827592.394874:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827592] req@ffff9a78841d92c0 x1874671476722432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827592.394889:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476722432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827592.394898:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827592.394903:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827592.394906:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722432:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827592.394913:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827592] req@ffff9a78841d9680 x1874671476722560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827592.394930:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476722560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827592.394944:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827592.394949:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827592.394954:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722560:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827592.394962:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827589/real 1787827592] req@ffff9a78841d8b40 x1874671476722176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827592.394975:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476722176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827592.394984:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827592.394987:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827592.394991:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476722176:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787827593.417998:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827594.442025:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827594.442033:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827594.442038:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827594.442066:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1961 00000100:00080000:1.0:1787827594.442077:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827594.442116:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827594.442151:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476722816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827594.442175:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827594.442175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827594.442182:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827594.442184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827594.442187:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1961 00000100:00080000:1.0:1787827594.442191:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827594.442194:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827594.442203:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827594.442211:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476722944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827594.442218:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476722816:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827594.442222:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827594.442226:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827594.442227:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827594.442230:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1961 00000100:00080000:1.0:1787827594.442232:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827594.442234:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827594.442237:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827594.442246:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476723072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827594.442252:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827594.442254:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827594.442256:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827594.442259:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1961 00000100:00080000:1.0:1787827594.442267:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827594.442271:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827594.442277:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827594.442285:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827594.442313:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476723200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827594.442322:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827594.442326:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827594.442328:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827594.442333:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1961 00000100:00080000:1.0:1787827594.442338:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827594.442339:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9a7890b1cf00 x1874671476722816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827594.442345:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827594.442367:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827594.442376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476722816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827594.442384:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476723328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827594.442399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827594.442405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827594.442410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476722816:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827594.442431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722944:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827594.442445:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827594.442456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9a7890b1d680 x1874671476722944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827594.442469:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476722944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827594.442477:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827594.442480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827594.442484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476722944:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827594.442490:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723072:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827594.442510:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827594.442522:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9a7890b1c3c0 x1874671476723072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827594.442532:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476723072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827594.442537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827594.442540:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827594.442543:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723072:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827594.442549:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723200:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827594.442568:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827594.442581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9a7890b1de00 x1874671476723200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827594.442593:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476723200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827594.442603:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827594.442606:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827594.442610:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723200:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827594.442616:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723328:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827594.442636:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827594.442647:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827594/real 1787827594] req@ffff9a7890b1e580 x1874671476723328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827594.442655:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476723328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827594.442662:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827594.442664:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827594.442666:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723328:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827595.466230:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827596.490001:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827597.514087:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827598.538109:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827599.561992:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827599.562004:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827599.562009:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827599.562039:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1967 00000100:00080000:1.0:1787827599.562060:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827599.562066:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827599.562077:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827599.562105:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476723456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827599.562129:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827599.562132:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827599.562134:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827599.562137:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1967 00000100:00080000:1.0:1787827599.562140:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827599.562144:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827599.562153:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827599.562163:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476723584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827599.562168:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476723456:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827599.562182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827599.562185:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827599.562187:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827599.562191:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1967 00000100:00080000:1.0:1787827599.562193:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827599.562203:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827599.562207:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827599.562217:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476723712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827599.562225:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827599.562228:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827599.562231:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787827599.562232:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827599.562237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1967 00000100:00080000:1.0:1787827599.562243:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827599.562246:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827599.562251:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827599.562263:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476723840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827599.562276:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827599.562280:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827599/real 1787827599] req@ffff9a7890b1f480 x1874671476723456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827599.562281:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827599.562285:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827599.562299:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1967 00000100:00080000:1.0:1787827599.562305:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827599.562311:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827599.562315:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827599.562321:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476723456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827599.562327:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476723968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827599.562370:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827599.562378:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827599.562385:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476723456:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827599.562420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723584:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827599.562440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827599.562450:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827599/real 1787827599] req@ffff9a7890b1f840 x1874671476723584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827599.562461:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476723584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827599.562481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827599.562485:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827599.562488:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723584:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827599.562495:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723712:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827599.562518:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827599.562529:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827599/real 1787827599] req@ffff9a7890b1da40 x1874671476723712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827599.562537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476723712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827599.562548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827599.562550:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827599.562553:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723712:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827599.562563:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723840:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827599.562577:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827599.562584:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827599/real 1787827599] req@ffff9a7890b1e940 x1874671476723840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827599.562592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476723840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827599.562598:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827599.562600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827599.562602:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723840:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827599.562615:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723968:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827599.562625:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827599.562631:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827599/real 1787827599] req@ffff9a7890b1f0c0 x1874671476723968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827599.562639:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476723968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827599.562644:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827599.562646:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827599.562649:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476723968:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827600.585998:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827601.610121:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827602.634085:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827603.658170:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827604.682054:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827604.682073:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827604.682082:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827604.682103:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1972 00010000:00010000:1.0:1787827604.682120:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827604.682125:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827604.682142:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827604.682219:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476724096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827604.682281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827604.682289:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827604.682293:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827604.682302:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1972 00000100:00080000:0.0:1787827604.682311:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827604.682321:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827604.682340:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827604.682357:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476724096:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827604.682370:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476724224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827604.682397:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827604.682407:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827604.682423:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827604.682448:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1972 00000100:00080000:0.0:1787827604.682454:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827604.682462:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827604.682470:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827604.682489:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476724352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827604.682507:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827604.682512:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827604.682520:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827604.682523:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827604.682531:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1972 00000100:00080000:0.0:1787827604.682537:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827604.682545:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827604.682568:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827604.682617:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476724480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827604.682638:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827604.682645:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827604/real 1787827604] req@ffff9a7890a3da40 x1874671476724096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827604.682648:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827604.682655:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827604.682680:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1972 00000100:00080000:0.0:1787827604.682695:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827604.682716:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476724096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827604.682720:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827604.682747:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787827604.682759:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827604.682772:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827604.682777:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476724608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827604.682784:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476724096:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827604.682855:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724224:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827604.682946:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827604.682966:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827604/real 1787827604] req@ffff9a7890a3fc00 x1874671476724224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827604.683037:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476724224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827604.683048:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827604.683052:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827604.683056:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724224:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827604.683064:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724352:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827604.683090:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827604.683105:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827604/real 1787827604] req@ffff9a7890a3de00 x1874671476724352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827604.683118:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476724352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827604.683127:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827604.683131:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827604.683135:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724352:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827604.683141:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724480:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827604.683159:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827604.683169:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827604/real 1787827604] req@ffff9a7890a3f480 x1874671476724480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827604.683192:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476724480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827604.683203:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827604.683206:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827604.683210:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724480:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827604.683215:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724608:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827604.683232:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827604.683241:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827604/real 1787827604] req@ffff9a7890a3c000 x1874671476724608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827604.683273:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476724608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827629 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827604.683291:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827604.683297:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827604.683303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724608:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827605.706047:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827606.730067:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827607.753996:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827608.778108:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827609.802038:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827609.802050:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827609.802056:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827609.802063:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827609.802069:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1977 00000100:00080000:2.0:1787827609.802085:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827609.802096:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827609.802146:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476724736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827609.802189:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827609.802193:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827609.802196:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827609.802202:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1977 00000100:00080000:2.0:1787827609.802209:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827609.802216:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827609.802230:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827609.802251:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476724864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827609.802269:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827609.802278:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827609.802282:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827609.802287:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1977 00000100:00080000:2.0:1787827609.802292:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827609.802296:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476724736:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827609.802302:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827609.802312:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827609.802328:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476724992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827609.802340:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827609.802344:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827609.802347:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827609.802351:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1977 00000100:00080000:2.0:1787827609.802367:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827609.802373:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827609.802380:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787827609.802384:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827609.802400:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476725120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827609.802410:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827609.802414:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827609.802416:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827609.802421:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1977 00000100:00080000:2.0:1787827609.802425:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827609.802430:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827609.802436:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827609.802443:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827609/real 1787827609] req@ffff9a78841d8b40 x1874671476724736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827609.802454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476725248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827609.802475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476724736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827609.802490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827609.802495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827609.802501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476724736:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827609.802508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724864:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827609.802529:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827609.802541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827609/real 1787827609] req@ffff9a78841d9680 x1874671476724864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827609.802552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476724864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827609.802560:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827609.802563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827609.802566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724864:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827609.802592:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724992:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827609.802608:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827609.802617:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827609/real 1787827609] req@ffff9a78841d92c0 x1874671476724992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827609.802628:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476724992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827609.802636:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827609.802639:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827609.802642:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476724992:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827609.802647:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725120:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827609.802661:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827609.802669:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827609/real 1787827609] req@ffff9a78841db840 x1874671476725120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827609.802681:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476725120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827609.802690:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827609.802693:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827609.802696:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725120:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827609.802701:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725248:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827609.802715:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827609.802724:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827609/real 1787827609] req@ffff9a78841da940 x1874671476725248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827609.802735:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476725248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827634 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827609.802744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827609.802746:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827609.802749:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725248:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827610.826112:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827611.850075:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827612.874116:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827613.897997:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827614.922035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827614.922047:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827614.922052:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827614.922064:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1982 00000100:00080000:1.0:1787827614.922076:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827614.922084:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827614.922088:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827614.922128:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476725376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827614.922158:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827614.922162:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827614.922165:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827614.922168:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1982 00000100:00080000:1.0:1787827614.922172:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827614.922176:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827614.922187:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827614.922207:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476725504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827614.922207:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476725376:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827614.922230:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827614.922234:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827614.922236:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827614.922240:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1982 00000100:00080000:1.0:1787827614.922242:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827614.922245:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827614.922250:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827614.922262:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476725632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827614.922271:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827614.922275:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827614.922276:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827614.922280:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1982 00000100:00080000:1.0:1787827614.922283:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827614.922291:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827614.922291:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827614.922302:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827614.922328:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476725760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827614.922358:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827614.922363:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827614.922367:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787827614.922367:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827614/real 1787827614] req@ffff9a7890b1fc00 x1874671476725376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827614.922380:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1982 00000100:00080000:1.0:1787827614.922390:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827614.922399:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827614.922407:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827614.922414:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476725376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827614.922428:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476725888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827614.922436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827614.922446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827614.922452:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476725376:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827614.922479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725504:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827614.922505:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827614.922518:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827614/real 1787827614] req@ffff9a7890b1c000 x1874671476725504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827614.922530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476725504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827614.922549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827614.922554:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827614.922571:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725504:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827614.922581:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725632:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827614.922612:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827614.922627:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827614/real 1787827614] req@ffff9a7890b1f0c0 x1874671476725632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827614.922639:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476725632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827614.922648:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827614.922653:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827614.922657:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725632:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827614.922666:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725760:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827614.922694:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827614.922711:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827614/real 1787827614] req@ffff9a7890b1e940 x1874671476725760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827614.922744:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476725760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827614.922753:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827614.922756:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827614.922760:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725760:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827614.922769:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725888:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827614.922799:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827614.922817:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827614/real 1787827614] req@ffff9a7890b1da40 x1874671476725888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827614.922836:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476725888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827639 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827614.922873:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827614.922876:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827614.922880:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476725888:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827615.946148:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827616.970104:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827617.994084:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827619.017995:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827620.041987:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827620.042004:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827620.042011:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827620.042028:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1987 00000100:00080000:1.0:1787827620.042057:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827620.042069:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827620.042086:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827620.042156:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476726016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827620.042192:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827620.042197:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827620.042199:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827620.042221:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1987 00000100:00080000:1.0:1787827620.042226:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827620.042233:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827620.042249:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827620.042268:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476726144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827620.042296:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827620.042304:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827620.042309:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787827620.042311:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476726016:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827620.042320:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1987 00000100:00080000:1.0:1787827620.042332:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827620.042337:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827620.042343:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827620.042377:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476726272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827620.042389:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827620.042393:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827620.042396:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827620.042400:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1987 00000100:00080000:1.0:1787827620.042404:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827620.042408:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827620.042414:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827620.042426:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476726400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827620.042436:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827620.042440:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827620.042442:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827620.042447:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1987 00000100:00080000:1.0:1787827620.042452:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787827620.042457:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827620.042461:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827620.042468:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827620.042483:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476726528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827620.042549:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff9a7890b1f840 x1874671476726016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827620.042583:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476726016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827620.042630:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827620.042638:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827620.042643:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476726016:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827620.042676:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726144:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827620.042717:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827620.042734:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff9a7890b1f480 x1874671476726144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827620.042748:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476726144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827620.042757:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827620.042760:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827620.042764:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726144:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827620.042770:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726272:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827620.042787:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827620.042796:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff9a7890b1e580 x1874671476726272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827620.042808:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476726272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827620.042816:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827620.042819:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827620.042822:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726272:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827620.042852:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726400:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827620.042894:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827620.042932:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff9a7890b1de00 x1874671476726400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827620.042947:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476726400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827620.042962:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827620.042966:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827620.042971:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726400:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827620.042979:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726528:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827620.043004:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827620.043023:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827620/real 1787827620] req@ffff9a7890b1c3c0 x1874671476726528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827620.043037:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476726528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827620.043049:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827620.043053:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827620.043056:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726528:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827621.065994:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827622.089950:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827623.113994:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827624.137979:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827625.162003:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827625.162016:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827625.162022:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827625.162035:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1992 00000100:00080000:2.0:1787827625.162047:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827625.162055:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827625.162094:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827625.162106:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476726656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827625.162137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827625.162140:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827625.162142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827625.162145:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1992 00000100:00080000:2.0:1787827625.162148:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827625.162152:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827625.162163:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827625.162173:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476726784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827625.162185:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476726656:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827625.162186:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827625.162196:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827625.162199:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827625.162204:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1992 00000100:00080000:2.0:1787827625.162208:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827625.162211:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827625.162214:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827625.162224:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476726912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827625.162231:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827625.162234:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827625.162235:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827625.162239:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1992 00000100:00080000:2.0:1787827625.162268:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827625.162271:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827625.162276:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827625.162284:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476727040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827625.162292:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787827625.162298:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787827625.162299:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827625.162303:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827625.162308:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1992 00000100:00080000:2.0:1787827625.162310:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827625.162313:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827625.162316:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827625.162325:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476727168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827625.162371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff9a78841da580 x1874671476726656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.162391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476726656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.162401:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.162405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827625.162409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476726656:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827625.162436:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726784:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827625.162457:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827625.162488:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff9a78841d8000 x1874671476726784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.162500:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476726784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.162508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.162511:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827625.162514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726784:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827625.162520:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726912:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827625.162536:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827625.162544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff9a78841db480 x1874671476726912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.162555:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476726912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.162562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.162565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827625.162568:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476726912:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827625.162572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727040:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827625.162587:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827625.162596:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff9a78841d9e00 x1874671476727040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.162607:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476727040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.162617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.162620:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827625.162622:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727040:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827625.162628:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727168:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827625.162641:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827625.162650:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827625/real 1787827625] req@ffff9a78841d83c0 x1874671476727168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.162660:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476727168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.162669:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.162672:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827625.162675:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727168:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827626.186100:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827627.210029:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827628.234032:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827629.257987:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827630.281922:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827630.281933:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827630.281938:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827630.281950:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 1997 00000100:00080000:2.0:1787827630.281962:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827630.281971:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827630.281972:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827630.282018:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476727296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827630.282044:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827630.282047:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827630.282049:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827630.282052:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1997 00000100:00080000:2.0:1787827630.282055:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827630.282058:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827630.282069:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827630.282082:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476727424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827630.282083:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476727296:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827630.282095:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827630.282100:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827630.282102:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827630.282105:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1997 00000100:00080000:2.0:1787827630.282107:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827630.282109:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827630.282113:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827630.282124:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476727552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827630.282130:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827630.282133:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827630.282135:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827630.282138:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1997 00000100:00080000:2.0:1787827630.282140:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827630.282142:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827630.282146:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827630.282154:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476727680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827630.282159:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827630.282162:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827630.282163:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827630.282167:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 1997 00000100:00080000:2.0:1787827630.282169:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827630.282171:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827630.282174:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787827630.282178:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827630.282185:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476727808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827630.282247:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff9a78841dbc00 x1874671476727296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827630.282263:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476727296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827630.282274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827630.282277:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827630.282281:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476727296:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827630.282313:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727424:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827630.282330:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827630.282340:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff9a78841da1c0 x1874671476727424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827630.282348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476727424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827630.282354:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827630.282357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827630.282359:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727424:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827630.282363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727552:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827630.282372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827630.282378:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff9a78841db0c0 x1874671476727552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827630.282385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476727552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827630.282391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827630.282393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827630.282396:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727552:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827630.282400:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727680:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827630.282411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827630.282416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff9a78841d9a40 x1874671476727680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827630.282423:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476727680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827630.282431:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827630.282433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827630.282434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727680:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827630.282438:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727808:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827630.282447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827630.282453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827630/real 1787827630] req@ffff9a78841d8780 x1874671476727808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827630.282460:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476727808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827630.282467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827630.282469:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827630.282471:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476727808:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827631.306225:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827632.330074:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827633.354022:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827634.378106:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827635.402007:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827635.402008:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827635.402016:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827635.402021:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827635.402032:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2002 00000100:00080000:0.0:1787827635.402043:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827635.402051:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827635.402097:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476727936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827635.402129:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827635.402132:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827635.402133:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827635.402136:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2002 00000100:00080000:0.0:1787827635.402139:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827635.402142:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827635.402150:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827635.402160:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3e580 x1874671476728064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827635.402168:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827635.402171:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827635.402172:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827635.402178:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2002 00000100:00100000:3.0:1787827635.402179:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476727936:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827635.402181:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827635.402188:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827635.402192:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827635.402219:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476728192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827635.402232:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827635.402235:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827635.402236:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827635.402239:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2002 00000100:00080000:0.0:1787827635.402244:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827635.402247:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827635.402251:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827635.402256:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827635.402265:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476728320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827635.402274:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827635.402287:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827635.402289:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827635.402292:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2002 00000100:00080000:0.0:1787827635.402294:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827635.402297:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827635.402300:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827635.402309:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476728448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827635.402317:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff9a7890a3c3c0 x1874671476727936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827635.402334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476727936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827635.402343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827635.402346:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827635.402349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476727936:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827635.402368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728064:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827635.402383:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827635.402392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff9a7890a3e580 x1874671476728064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827635.402400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3e580 x1874671476728064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827635.402406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827635.402408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827635.402410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728064:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827635.402414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728192:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827635.402425:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827635.402431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff9a7890a3c000 x1874671476728192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827635.402439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476728192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827635.402449:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827635.402452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827635.402454:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728192:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827635.402458:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728320:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827635.402469:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827635.402475:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff9a7890a3f480 x1874671476728320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827635.402482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476728320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827635.402489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827635.402491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827635.402493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728320:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827635.402498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728448:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827635.402507:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827635.402513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827635/real 1787827635] req@ffff9a7890a3de00 x1874671476728448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827635.402520:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476728448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827635.402526:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827635.402528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827635.402531:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728448:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827636.426006:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827637.450016:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827638.474008:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827639.498060:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827640.521968:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827640.521976:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827640.521981:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827640.521989:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2008 00000100:00080000:2.0:1787827640.521998:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827640.522004:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827640.522020:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827640.522038:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476728576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827640.522058:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827640.522061:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827640.522063:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827640.522065:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2008 00000100:00080000:2.0:1787827640.522068:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827640.522070:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827640.522089:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827640.522098:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476728704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827640.522106:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827640.522111:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787827640.522112:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476728576:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827640.522113:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827640.522120:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2008 00000100:00080000:2.0:1787827640.522124:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827640.522126:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827640.522129:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827640.522139:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476728832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827640.522156:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827640.522160:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827640.522161:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827640.522165:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2008 00000100:00080000:2.0:1787827640.522170:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827640.522174:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827640.522181:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827640.522182:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827640.522198:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476728960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827640.522217:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827640.522219:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827640.522221:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827640.522224:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2008 00000100:00080000:2.0:1787827640.522227:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827640.522230:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827640.522234:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827640.522243:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476729088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827640.522250:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827640/real 1787827640] req@ffff9a78841d8780 x1874671476728576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827640.522282:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476728576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827640.522294:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827640.522299:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827640.522303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476728576:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827640.522326:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728704:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827640.522348:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827640.522370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827640/real 1787827640] req@ffff9a78841d9a40 x1874671476728704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827640.522384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476728704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827640.522403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827640.522406:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827640.522410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728704:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827640.522415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728832:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827640.522435:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827640.522455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827640/real 1787827640] req@ffff9a78841db0c0 x1874671476728832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827640.522468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476728832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827640.522476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827640.522480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827640.522483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728832:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827640.522490:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728960:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827640.522508:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827640.522518:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827640/real 1787827640] req@ffff9a78841da1c0 x1874671476728960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827640.522529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476728960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827640.522539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827640.522543:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827640.522546:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476728960:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827640.522551:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729088:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827640.522566:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827640.522575:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827640/real 1787827640] req@ffff9a78841dbc00 x1874671476729088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827640.522596:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476729088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827640.522607:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827640.522610:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827640.522613:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729088:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827641.546090:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827642.570003:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827643.594005:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827644.618019:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827645.641910:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827645.641919:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827645.641922:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827645.641930:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2013 00000100:00080000:2.0:1787827645.641939:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827645.641946:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827645.641976:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827645.641978:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476729216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827645.642006:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827645.642009:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827645.642011:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827645.642014:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2013 00000100:00080000:2.0:1787827645.642017:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827645.642021:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827645.642029:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827645.642034:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476729216:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827645.642041:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476729344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827645.642049:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827645.642052:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827645.642053:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827645.642056:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2013 00000100:00080000:2.0:1787827645.642059:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827645.642062:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827645.642065:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827645.642074:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476729472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827645.642081:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827645.642083:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827645.642085:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827645.642088:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2013 00000100:00080000:2.0:1787827645.642091:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787827645.642093:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787827645.642096:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827645.642101:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827645.642108:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476729600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827645.642115:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827645.642118:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827645.642119:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827645.642121:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2013 00000100:00080000:2.0:1787827645.642123:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827645.642125:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827645.642128:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827645.642144:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476729728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827645.642149:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827645/real 1787827645] req@ffff9a78841d83c0 x1874671476729216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827645.642168:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476729216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827645.642177:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827645.642180:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827645.642182:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476729216:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827645.642200:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729344:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827645.642218:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827645.642227:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827645/real 1787827645] req@ffff9a78841d9e00 x1874671476729344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827645.642235:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476729344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827645.642241:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827645.642243:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827645.642245:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729344:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827645.642250:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729472:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827645.642260:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827645.642266:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827645/real 1787827645] req@ffff9a78841db480 x1874671476729472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827645.642273:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476729472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827645.642278:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827645.642280:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827645.642282:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729472:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827645.642285:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729600:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827645.642295:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827645.642301:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827645/real 1787827645] req@ffff9a78841d8000 x1874671476729600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827645.642308:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476729600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827645.642314:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827645.642316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827645.642318:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729600:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827645.642321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729728:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827645.642330:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827645.642336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827645/real 1787827645] req@ffff9a78841da580 x1874671476729728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827645.642343:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476729728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827645.642349:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827645.642351:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827645.642353:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729728:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827646.665987:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827647.690004:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827648.714090:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827649.737967:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827650.761963:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827650.761974:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827650.761980:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827650.761991:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2018 00000100:00080000:2.0:1787827650.762004:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827650.762011:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827650.762042:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827650.762058:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476729856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827650.762084:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827650.762088:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827650.762090:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827650.762095:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2018 00000100:00080000:2.0:1787827650.762099:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827650.762105:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827650.762118:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827650.762139:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476729984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827650.762139:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476729856:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827650.762165:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827650.762168:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827650.762170:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827650.762176:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2018 00000100:00080000:2.0:1787827650.762180:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827650.762183:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827650.762188:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827650.762201:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476730112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827650.762212:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787827650.762219:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787827650.762222:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827650.762227:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827650.762236:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2018 00000100:00080000:2.0:1787827650.762242:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827650.762249:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827650.762256:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827650.762266:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827650/real 1787827650] req@ffff9a78841da940 x1874671476729856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827650.762273:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476730240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827650.762303:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827650.762307:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787827650.762308:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476729856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827650.762311:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827650.762323:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2018 00000100:00080000:2.0:1787827650.762340:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827650.762343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827650.762349:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787827650.762352:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:2.0:1787827650.762360:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827650.762361:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476729856:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827650.762378:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476730368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827650.762393:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729984:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827650.762420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827650.762430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827650/real 1787827650] req@ffff9a78841db840 x1874671476729984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827650.762438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476729984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827650.762448:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827650.762450:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827650.762453:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476729984:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827650.762457:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730112:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827650.762468:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827650.762473:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827650/real 1787827650] req@ffff9a78841d92c0 x1874671476730112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827650.762480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476730112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827650.762485:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827650.762487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827650.762489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730112:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827650.762493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730240:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827650.762503:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827650.762508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827650/real 1787827650] req@ffff9a78841d9680 x1874671476730240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827650.762515:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476730240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827650.762521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827650.762523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827650.762526:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730240:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827650.762530:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730368:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827650.762540:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827650.762545:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827650/real 1787827650] req@ffff9a78841d8b40 x1874671476730368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827650.762552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476730368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827650.762558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827650.762560:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827650.762562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730368:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827651.785978:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827652.809966:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827653.834013:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827654.857979:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827655.881966:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827655.881978:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827655.881982:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827655.881994:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2023 00000100:00080000:2.0:1787827655.882006:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827655.882015:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827655.882040:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827655.882063:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476730496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827655.882089:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827655.882094:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827655.882096:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827655.882098:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2023 00000100:00080000:2.0:1787827655.882101:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827655.882104:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827655.882114:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827655.882124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476730624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827655.882134:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476730496:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827655.882140:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827655.882147:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827655.882149:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827655.882153:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2023 00000100:00080000:2.0:1787827655.882155:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827655.882159:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827655.882163:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827655.882173:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476730752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827655.882180:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827655.882184:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827655.882186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827655.882189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2023 00000100:00080000:2.0:1787827655.882203:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827655.882206:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827655.882211:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827655.882221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476730880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827655.882228:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827655.882231:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827655.882233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827655.882237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2023 00000100:00080000:2.0:1787827655.882242:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827655.882247:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827655.882251:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827655.882259:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476731008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827655.882265:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730624:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827655.882306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730752:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827655.882321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730880:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827655.882338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731008:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827656.906007:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827657.930093:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827658.954082:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827658.954150:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787827658.954184:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827658.954197:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827658.954235:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827658.954245:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827658.954255:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827658.954265:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827658.954287:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827655/real 1787827658] req@ffff9a78841da940 x1874671476731008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827658.954314:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476731008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827658.954329:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827658.954334:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827658.954339:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731008:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827658.954348:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827655/real 1787827658] req@ffff9a78841d9680 x1874671476730624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827658.954363:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476730624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827658.954372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827658.954376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827658.954381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730624:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827658.954389:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827655/real 1787827658] req@ffff9a78841d92c0 x1874671476730752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827658.954419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476730752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827658.954428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827658.954432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827658.954436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730752:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827658.954443:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827655/real 1787827658] req@ffff9a78841db840 x1874671476730880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827658.954456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476730880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827658.954467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827658.954470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827658.954474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476730880:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827658.954481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827655/real 1787827658] req@ffff9a78841d8b40 x1874671476730496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827658.954493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476730496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827658.954501:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827658.954504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827658.954508:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476730496:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787827659.978025:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827661.001913:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827661.001921:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827661.001925:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827661.001934:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2028 00000100:00080000:2.0:1787827661.001942:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827661.001948:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827661.001981:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476731136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827661.002003:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827661.002006:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827661.002007:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827661.002010:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2028 00000100:00080000:2.0:1787827661.002013:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827661.002016:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827661.002023:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827661.002033:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476731264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827661.002041:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827661.002044:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827661.002045:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827661.002050:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476731136:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827661.002051:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2028 00000100:00080000:2.0:1787827661.002058:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827661.002060:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827661.002063:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827661.002072:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476731392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827661.002078:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827661.002081:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827661.002082:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827661.002085:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2028 00000100:00080000:2.0:1787827661.002098:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827661.002101:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827661.002105:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827661.002109:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827661.002118:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476731520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827661.002124:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827661.002133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827661.002136:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827661.002138:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827661.002140:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2028 00000100:00080000:2.0:1787827661.002143:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827661.002147:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827661.002151:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff9a78841da580 x1874671476731136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787827661.002151:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827661.002164:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476731648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827661.002172:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476731136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827661.002181:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827661.002185:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827661.002188:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476731136:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827661.002208:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731264:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827661.002238:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827661.002258:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff9a78841d8000 x1874671476731264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827661.002278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476731264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827661.002287:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827661.002290:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827661.002294:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731264:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827661.002300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731392:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827661.002321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827661.002334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff9a78841db480 x1874671476731392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827661.002347:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476731392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827661.002355:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827661.002359:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827661.002362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731392:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827661.002368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731520:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827661.002397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827661.002414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff9a78841d9e00 x1874671476731520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827661.002425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476731520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827661.002440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827661.002444:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827661.002446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731520:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827661.002452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731648:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827661.002472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827661.002485:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827661/real 1787827661] req@ffff9a78841d83c0 x1874671476731648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827661.002495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476731648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827661.002505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827661.002513:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827661.002517:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731648:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827662.025983:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827663.050110:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827664.073992:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827665.097966:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827666.121941:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827666.121949:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827666.121953:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827666.121964:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2033 00000100:00080000:2.0:1787827666.121976:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827666.121984:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827666.122040:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476731776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787827666.122048:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827666.122062:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827666.122067:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827666.122069:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827666.122074:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2033 00000100:00080000:2.0:1787827666.122078:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827666.122079:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476731776:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827666.122086:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827666.122097:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827666.122111:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476731904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827666.122119:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827666.122122:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827666.122124:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827666.122127:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2033 00000100:00080000:2.0:1787827666.122130:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827666.122133:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827666.122137:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787827666.122137:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827666.122148:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476732032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827666.122156:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827666.122159:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827666.122160:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827666.122162:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2033 00000100:00080000:2.0:1787827666.122165:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827666.122168:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827666.122173:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827666.122181:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff9a78841dbc00 x1874671476731776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827666.122190:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476732160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827666.122198:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827666.122199:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476731776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827666.122203:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827666.122207:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827666.122215:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2033 00000100:00080000:2.0:1787827666.122221:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827666.122226:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:1.0:1787827666.122230:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:2.0:1787827666.122230:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:1.0:1787827666.122237:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787827666.122243:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476732288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827666.122244:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476731776:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827666.122271:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731904:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827666.122288:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827666.122296:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff9a78841da1c0 x1874671476731904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827666.122305:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476731904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827666.122310:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827666.122313:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827666.122315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476731904:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827666.122319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732032:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827666.122329:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827666.122334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff9a78841db0c0 x1874671476732032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827666.122342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476732032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827666.122357:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827666.122360:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827666.122365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732032:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827666.122371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732160:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827666.122392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827666.122404:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff9a78841d9a40 x1874671476732160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827666.122417:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476732160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827666.122424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827666.122426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827666.122428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732160:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827666.122433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732288:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827666.122447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827666.122454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827666/real 1787827666] req@ffff9a78841d8780 x1874671476732288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827666.122461:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476732288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827666.122467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827666.122469:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827666.122472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732288:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827667.145973:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827668.169992:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827669.193976:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827670.218057:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827671.241939:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827671.241940:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827671.241951:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827671.241956:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827671.241975:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2038 00000100:00080000:1.0:1787827671.241984:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827671.241990:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827671.242022:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476732416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827671.242048:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827671.242052:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827671.242055:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827671.242061:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2038 00000100:00080000:1.0:1787827671.242066:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827671.242071:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827671.242079:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476732416:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787827671.242081:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827671.242099:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476732544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827671.242110:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827671.242113:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827671.242114:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827671.242118:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2038 00000100:00080000:1.0:1787827671.242131:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827671.242136:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827671.242140:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827671.242142:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827671.242173:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476732672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827671.242179:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9a7890b1d680 x1874671476732416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827671.242189:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827671.242193:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827671.242195:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827671.242200:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2038 00000100:00100000:0.0:1787827671.242201:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476732416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827671.242204:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827671.242211:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787827671.242216:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787827671.242218:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:0.0:1787827671.242222:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827671.242226:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476732416:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827671.242231:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476732800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827671.242239:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827671.242241:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827671.242242:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827671.242246:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2038 00000100:00080000:1.0:1787827671.242248:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827671.242250:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732544:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787827671.242253:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827671.242256:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827671.242264:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476732928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827671.242280:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827671.242292:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9a7890b1cf00 x1874671476732544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827671.242308:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476732544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827671.242317:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827671.242320:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827671.242323:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732544:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827671.242330:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732672:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827671.242350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827671.242362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9a7890b1c3c0 x1874671476732672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827671.242372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476732672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827671.242377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827671.242380:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827671.242382:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732672:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827671.242386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732800:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827671.242400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827671.242407:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9a7890b1de00 x1874671476732800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827671.242415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476732800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827671.242425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827671.242427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827671.242430:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732800:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827671.242440:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732928:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827671.242450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827671.242456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827671/real 1787827671] req@ffff9a7890b1e580 x1874671476732928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827671.242463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476732928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827671.242470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827671.242471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827671.242473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476732928:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827672.266009:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827673.289976:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827674.313964:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827675.338000:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827676.361961:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827676.361969:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827676.361973:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827676.361983:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2043 00000100:00080000:2.0:1787827676.361993:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827676.362000:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827676.362024:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827676.362038:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476733056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827676.362064:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827676.362067:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827676.362069:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827676.362071:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2043 00000100:00080000:2.0:1787827676.362073:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827676.362076:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827676.362084:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827676.362093:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476733184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827676.362105:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827676.362115:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827676.362116:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827676.362119:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2043 00000100:00080000:2.0:1787827676.362121:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827676.362134:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827676.362136:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476733056:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787827676.362138:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827676.362149:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476733312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827676.362158:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827676.362160:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827676.362161:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827676.362163:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2043 00000100:00080000:2.0:1787827676.362166:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827676.362169:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827676.362172:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827676.362180:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476733440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827676.362187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827676.362189:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827676.362191:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827676.362194:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2043 00000100:00080000:2.0:1787827676.362197:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827676.362199:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827676.362202:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827676.362209:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476733568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827676.362251:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827676.362323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827676] req@ffff9a78841d8780 x1874671476733056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827676.362344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476733056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827676.362356:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827676.362361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827676.362365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476733056:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827676.362373:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733184:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827676.362405:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827676.362418:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827676] req@ffff9a78841d9a40 x1874671476733184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827676.362429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476733184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827676.362435:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827676.362438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827676.362441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733184:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827676.362469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733312:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827676.362505:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827676.362517:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827676] req@ffff9a78841db0c0 x1874671476733312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827676.362528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476733312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827676.362535:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827676.362537:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827676.362540:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733312:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827676.362545:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733440:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827676.362570:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827676.362578:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827676] req@ffff9a78841da1c0 x1874671476733440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827676.362588:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476733440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827676.362597:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827676.362600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827676.362603:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733440:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827676.362609:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733568:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827676.362622:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827676.362630:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827676/real 1787827676] req@ffff9a78841dbc00 x1874671476733568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827676.362639:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476733568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827676.362647:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827676.362649:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827676.362652:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733568:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827677.385997:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827678.410010:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827679.434048:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827680.458033:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827681.482036:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827681.482036:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827681.482050:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827681.482057:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827681.482071:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2048 00000100:00080000:0.0:1787827681.482084:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827681.482093:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827681.482140:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476733696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827681.482181:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827681.482186:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827681.482189:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827681.482194:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2048 00000100:00080000:0.0:1787827681.482198:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827681.482202:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827681.482214:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827681.482231:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476733824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827681.482246:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827681.482251:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827681.482254:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827681.482260:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2048 00000100:00080000:0.0:1787827681.482263:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827681.482274:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827681.482276:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476733696:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787827681.482281:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827681.482304:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476733952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827681.482327:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827681.482332:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827681.482335:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827681.482339:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2048 00000100:00080000:0.0:1787827681.482363:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827681.482367:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827681.482382:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787827681.482389:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827681.482416:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476734080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827681.482429:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827681.482434:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827681.482437:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827681.482441:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2048 00000100:00080000:0.0:1787827681.482476:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827681.482483:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827681.482492:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827681.482508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827681/real 1787827681] req@ffff9a7890a3fc00 x1874671476733696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827681.482513:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476734208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827681.482569:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476733696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827681.482589:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827681.482594:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827681.482600:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476733696:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827681.482611:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733824:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827681.482644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827681.482661:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827681/real 1787827681] req@ffff9a7890a3da40 x1874671476733824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827681.482677:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476733824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827681.482687:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827681.482691:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827681.482696:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733824:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827681.482726:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733952:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827681.482745:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827681.482758:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827681/real 1787827681] req@ffff9a7890a3de00 x1874671476733952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827681.482773:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476733952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827681.482782:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827681.482786:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827681.482790:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476733952:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827681.482797:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734080:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827681.482815:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827681.482826:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827681/real 1787827681] req@ffff9a7890a3f480 x1874671476734080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827681.482887:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476734080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827681.482900:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827681.482904:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827681.482908:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734080:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827681.482915:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734208:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827681.482939:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827681.482953:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827681/real 1787827681] req@ffff9a7890a3c000 x1874671476734208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827681.482966:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476734208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827681.482986:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827681.482990:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827681.483008:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734208:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827682.506070:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827683.530066:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827684.554094:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827685.577998:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827686.601910:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827686.601920:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827686.601924:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827686.601933:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2054 00000100:00080000:0.0:1787827686.601943:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827686.601949:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827686.601954:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827686.601979:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3e580 x1874671476734336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827686.601999:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827686.602001:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827686.602002:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827686.602006:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2054 00000100:00080000:0.0:1787827686.602008:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827686.602011:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827686.602019:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827686.602027:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476734336:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827686.602030:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476734464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827686.602038:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827686.602041:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827686.602042:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827686.602045:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2054 00000100:00080000:0.0:1787827686.602047:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827686.602049:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827686.602052:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827686.602062:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476734592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827686.602068:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827686.602071:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827686.602073:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827686.602075:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2054 00000100:00080000:0.0:1787827686.602078:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827686.602080:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827686.602085:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787827686.602087:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827686.602096:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476734720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827686.602103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827686.602105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827686.602107:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827686.602110:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2054 00000100:00080000:0.0:1787827686.602112:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827686.602114:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827686.602118:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827686.602127:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476734848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827686.602131:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827686/real 1787827686] req@ffff9a7890a3e580 x1874671476734336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827686.602148:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3e580 x1874671476734336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827686.602157:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827686.602160:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827686.602162:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476734336:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827686.602180:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734464:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827686.602216:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827686.602229:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827686/real 1787827686] req@ffff9a7890a3c3c0 x1874671476734464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827686.602241:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476734464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827686.602258:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827686.602262:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827686.602268:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734464:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827686.602277:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734592:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827686.602317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827686.602331:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827686/real 1787827686] req@ffff9a7890a3c000 x1874671476734592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827686.602345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476734592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827686.602352:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827686.602354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827686.602357:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734592:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827686.602361:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734720:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827686.602379:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827686.602387:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827686/real 1787827686] req@ffff9a7890a3f480 x1874671476734720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827686.602396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476734720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827686.602403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827686.602405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827686.602446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734720:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827686.602451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734848:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827686.602466:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827686.602474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827686/real 1787827686] req@ffff9a7890a3de00 x1874671476734848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827686.602482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476734848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827686.602488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827686.602490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827686.602492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476734848:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827687.625960:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827688.650013:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827689.673957:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827690.697967:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827691.721947:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827691.721958:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827691.721963:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827691.721975:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2059 00000100:00080000:1.0:1787827691.721987:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827691.721993:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827691.721995:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827691.722045:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476734976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827691.722072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827691.722075:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827691.722077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827691.722080:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2059 00000100:00080000:1.0:1787827691.722083:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827691.722087:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827691.722097:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827691.722106:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476735104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827691.722111:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476734976:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827691.722118:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827691.722121:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827691.722123:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827691.722126:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2059 00000100:00080000:1.0:1787827691.722128:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827691.722130:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827691.722134:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827691.722142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476735232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827691.722148:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827691.722150:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827691.722151:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827691.722154:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2059 00000100:00080000:1.0:1787827691.722196:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827691.722201:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827691.722209:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827691.722212:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827691.722228:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476735360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827691.722259:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827691.722264:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827691.722266:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827691.722270:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2059 00000100:00080000:1.0:1787827691.722275:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827691.722280:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827691.722287:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827691.722323:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476735488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827691.722336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9a7890b1f480 x1874671476734976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827691.722369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476734976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827691.722423:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827691.722429:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827691.722435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476734976:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827691.722471:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735104:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827691.722519:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827691.722529:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9a7890b1f840 x1874671476735104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827691.722538:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476735104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827691.722544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827691.722546:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827691.722548:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735104:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827691.722552:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735232:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827691.722563:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827691.722568:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9a7890b1da40 x1874671476735232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827691.722576:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476735232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827691.722581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827691.722582:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827691.722585:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735232:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827691.722589:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735360:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827691.722600:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827691.722605:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9a7890b1e940 x1874671476735360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827691.722613:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476735360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827691.722620:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827691.722621:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827691.722623:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735360:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827691.722627:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735488:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827691.722637:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827691.722642:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9a7890b1f0c0 x1874671476735488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827691.722650:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476735488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827691.722656:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827691.722657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827691.722659:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735488:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827692.745969:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827693.770162:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827694.793971:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827695.817996:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827696.841922:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827696.841931:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827696.841935:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827696.841945:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2064 00000100:00080000:1.0:1787827696.841956:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827696.841963:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827696.841986:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827696.842002:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476735616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827696.842029:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827696.842033:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827696.842036:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827696.842043:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2064 00000100:00080000:1.0:1787827696.842049:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827696.842055:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827696.842062:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476735616:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787827696.842071:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827696.842091:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476735744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827696.842103:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827696.842106:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827696.842108:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827696.842112:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2064 00000100:00080000:1.0:1787827696.842117:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827696.842121:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827696.842125:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787827696.842128:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827696.842150:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476735872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827696.842160:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827696.842163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827696.842166:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827696.842169:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2064 00000100:00080000:1.0:1787827696.842173:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827696.842175:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9a7890b1c000 x1874671476735616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827696.842180:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827696.842191:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827696.842202:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476735616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827696.842212:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476736000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827696.842216:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827696.842226:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827696.842235:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476735616:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827696.842261:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827696.842264:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827696.842266:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827696.842272:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2064 00000100:00100000:0.0:1787827696.842276:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735744:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787827696.842277:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827696.842285:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827696.842291:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:0.0:1787827696.842305:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827696.842307:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476736128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827696.842321:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9a7890b1fc00 x1874671476735744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827696.842336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476735744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827696.842359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827696.842362:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827696.842365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735744:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827696.842370:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735872:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827696.842385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827696.842394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9a7890b1f0c0 x1874671476735872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827696.842408:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476735872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827696.842414:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827696.842417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827696.842419:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476735872:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827696.842423:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736000:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827696.842435:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827696.842442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9a7890b1e940 x1874671476736000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827696.842451:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476736000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827696.842459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827696.842461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827696.842463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736000:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827696.842475:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736128:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827696.842487:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827696.842494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9a7890b1da40 x1874671476736128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827696.842502:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476736128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827696.842515:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827696.842517:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827696.842520:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736128:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827697.865986:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827698.890001:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827699.913999:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827700.938059:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827701.961930:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827701.961938:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827701.961942:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827701.961951:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2069 00010000:00010000:2.0:1787827701.961954:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827701.961960:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827701.961968:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827701.962000:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476736256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827701.962021:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827701.962024:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827701.962025:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827701.962028:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2069 00000100:00080000:0.0:1787827701.962031:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827701.962035:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827701.962042:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827701.962048:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476736256:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827701.962054:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476736384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827701.962062:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827701.962065:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827701.962067:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827701.962069:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2069 00000100:00080000:0.0:1787827701.962071:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827701.962074:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827701.962077:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827701.962087:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476736512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827701.962092:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827701.962095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827701.962096:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827701.962099:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2069 00000100:00080000:0.0:1787827701.962101:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827701.962103:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827701.962108:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827701.962109:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827701.962118:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476736640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827701.962124:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827701.962127:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827701.962128:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827701.962132:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2069 00000100:00080000:0.0:1787827701.962134:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827701.962136:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827701.962139:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827701.962147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476736768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827701.962161:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827701/real 1787827701] req@ffff9a7890a3da40 x1874671476736256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827701.962176:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476736256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827701.962184:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827701.962187:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827701.962190:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476736256:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827701.962209:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736384:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827701.962226:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827701.962234:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827701/real 1787827701] req@ffff9a7890a3fc00 x1874671476736384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827701.962243:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476736384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827701.962249:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827701.962251:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827701.962253:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736384:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827701.962256:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736512:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827701.962266:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827701.962271:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827701/real 1787827701] req@ffff9a7890a3de00 x1874671476736512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827701.962279:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476736512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827701.962283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827701.962285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827701.962287:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736512:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827701.962290:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736640:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827701.962315:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827701.962324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827701/real 1787827701] req@ffff9a7890a3f480 x1874671476736640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827701.962332:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476736640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827701.962343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827701.962345:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827701.962348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736640:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827701.962351:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736768:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827701.962364:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827701.962370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827701/real 1787827701] req@ffff9a7890a3c000 x1874671476736768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827701.962376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476736768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827701.962382:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827701.962384:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827701.962386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476736768:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827702.986062:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827704.010013:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827705.034052:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827706.058043:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827707.081931:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827707.081941:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827707.081945:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827707.081955:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2074 00000100:00080000:1.0:1787827707.081964:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827707.081964:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827707.081973:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827707.082003:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476736896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827707.082026:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827707.082029:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827707.082030:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827707.082033:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2074 00000100:00080000:1.0:1787827707.082036:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827707.082040:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827707.082047:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827707.082058:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476737024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827707.082060:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476736896:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827707.082075:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827707.082077:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827707.082079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827707.082082:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2074 00000100:00080000:1.0:1787827707.082084:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827707.082086:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827707.082089:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827707.082098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476737152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827707.082105:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827707.082107:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827707.082108:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827707.082111:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2074 00000100:00080000:1.0:1787827707.082113:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827707.082115:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827707.082120:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827707.082123:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827707.082130:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476737280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827707.082137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827707.082139:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827707.082140:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827707.082143:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2074 00000100:00080000:1.0:1787827707.082145:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827707.082147:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827707.082151:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827707.082160:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476737408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827707.082166:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff9a7890b1f840 x1874671476736896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827707.082182:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476736896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827707.082190:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827707.082193:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827707.082196:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476736896:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827707.082214:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737024:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827707.082228:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827707.082236:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff9a7890b1f480 x1874671476737024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827707.082244:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476737024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827707.082250:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827707.082252:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827707.082254:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737024:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827707.082258:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737152:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827707.082283:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827707.082290:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff9a7890b1e580 x1874671476737152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827707.082298:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476737152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827707.082303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827707.082305:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827707.082307:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737152:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827707.082311:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737280:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827707.082321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827707.082326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff9a7890b1de00 x1874671476737280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827707.082333:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476737280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827707.082339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827707.082341:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827707.082343:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737280:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827707.082346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737408:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827707.082355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827707.082361:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827707/real 1787827707] req@ffff9a7890b1c3c0 x1874671476737408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827707.082368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476737408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827707.082374:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827707.082376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827707.082378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737408:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827708.106072:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827709.129959:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827710.153919:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827711.178028:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827712.201941:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827712.201959:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827712.201964:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827712.201974:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2079 00000100:00080000:1.0:1787827712.201985:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827712.201992:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827712.202015:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827712.202029:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476737536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827712.202053:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827712.202057:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827712.202059:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827712.202062:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2079 00000100:00080000:1.0:1787827712.202065:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827712.202069:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827712.202078:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827712.202089:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476737664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827712.202100:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827712.202104:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827712.202106:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827712.202112:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476737536:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827712.202113:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2079 00000100:00080000:1.0:1787827712.202119:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827712.202125:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827712.202129:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827712.202142:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476737792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827712.202159:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827712.202162:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827712.202164:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827712.202168:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2079 00000100:00080000:1.0:1787827712.202172:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827712.202175:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827712.202180:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827712.202184:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827712.202196:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476737920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827712.202212:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827712.202215:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827712.202217:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827712.202227:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2079 00000100:00080000:1.0:1787827712.202250:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827712.202252:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff9a7890b1cf00 x1874671476737536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827712.202258:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827712.202269:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827712.202302:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476737536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827712.202308:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476738048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827712.202320:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827712.202332:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827712.202341:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476737536:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827712.202371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737664:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827712.202399:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827712.202412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff9a7890b1d680 x1874671476737664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827712.202425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476737664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827712.202440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827712.202443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827712.202446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737664:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827712.202452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737792:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827712.202467:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827712.202476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff9a7890b1c3c0 x1874671476737792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827712.202487:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476737792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827712.202495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827712.202497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827712.202501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737792:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827712.202506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737920:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827712.202520:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827712.202528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff9a7890b1de00 x1874671476737920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827712.202538:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476737920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827712.202548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827712.202550:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827712.202554:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476737920:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827712.202559:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738048:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827712.202573:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827712.202581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827712/real 1787827712] req@ffff9a7890b1e580 x1874671476738048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827712.202608:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476738048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827712.202616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827712.202618:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827712.202620:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738048:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827713.226006:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827714.249990:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827715.274092:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827716.298108:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827717.322021:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827717.322039:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827717.322046:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827717.322063:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2084 00000100:00080000:1.0:1787827717.322076:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827717.322087:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827717.322138:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476738176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827717.322150:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827717.322172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827717.322177:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827717.322179:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827717.322182:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2084 00000100:00080000:1.0:1787827717.322185:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827717.322188:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827717.322215:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827717.322239:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476738304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827717.322248:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827717.322251:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827717.322253:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827717.322258:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2084 00000100:00080000:1.0:1787827717.322262:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827717.322266:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827717.322270:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827717.322282:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476738176:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827717.322284:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476738432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827717.322305:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827717.322321:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827717.322323:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827717.322327:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2084 00000100:00080000:1.0:1787827717.322330:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827717.322332:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827717.322337:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827717.322350:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476738560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827717.322357:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827717.322361:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827717.322363:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827717.322366:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2084 00000100:00080000:1.0:1787827717.322414:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827717.322421:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827717.322431:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827717.322432:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827717.322463:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476738688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827717.322525:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9a7890b1f480 x1874671476738176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827717.322552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476738176/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827717.322571:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827717.322578:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827717.322602:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476738176:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827717.322615:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738304:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827717.322668:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827717.322684:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9a7890b1f840 x1874671476738304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827717.322696:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476738304/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827717.322704:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827717.322707:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827717.322710:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738304:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827717.322760:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738432:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827717.322792:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827717.322812:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9a7890b1da40 x1874671476738432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827717.322829:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476738432/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827717.322841:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827717.322846:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827717.322851:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738432:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827717.322862:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738560:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827717.322896:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827717.322931:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9a7890b1e940 x1874671476738560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827717.322996:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476738560/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827717.323014:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827717.323020:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827717.323027:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738560:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827717.323038:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738688:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827717.323086:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827717.323105:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827717/real 1787827717] req@ffff9a7890b1f0c0 x1874671476738688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827717.323124:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476738688/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827717.323140:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827717.323145:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827717.323149:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738688:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827718.346101:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827719.370033:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827720.394025:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827721.417971:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827722.442032:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827722.442044:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827722.442080:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827722.442093:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2089 00000100:00080000:2.0:1787827722.442106:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827722.442127:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827722.442132:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827722.442192:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476738816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827722.442249:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827722.442252:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827722.442254:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827722.442257:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2089 00000100:00080000:2.0:1787827722.442260:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827722.442265:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827722.442275:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827722.442300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476738816:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827722.442308:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476738944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827722.442319:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827722.442322:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827722.442325:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827722.442329:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2089 00000100:00080000:2.0:1787827722.442332:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827722.442337:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827722.442342:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827722.442371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476739072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827722.442379:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827722.442382:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827722.442383:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827722.442387:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2089 00000100:00080000:2.0:1787827722.442390:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827722.442393:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827722.442397:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827722.442408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476739200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827722.442416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827722.442419:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827722.442421:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827722.442427:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2089 00000100:00080000:2.0:1787827722.442429:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827722.442442:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827722.442447:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827722.442456:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476739328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827722.442468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738944:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827722.442503:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739072:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827722.442523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739200:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827722.442545:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739328:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827723.465991:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827724.490012:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827725.514098:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827725.514113:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787827725.514160:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827725.514173:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827725.514202:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827725.514209:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827725.514219:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827725.514231:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827725.514236:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827725] req@ffff9a78841d9e00 x1874671476738944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827725.514271:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476738944/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827725.514282:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827725.514287:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827725.514292:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476738944:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827725.514301:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827725] req@ffff9a78841db480 x1874671476739072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827725.514315:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476739072/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827725.514323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827725.514328:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827725.514331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739072:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827725.514338:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827725] req@ffff9a78841d8000 x1874671476739200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827725.514352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476739200/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827725.514362:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827725.514365:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827725.514368:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739200:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827725.514375:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827725] req@ffff9a78841d83c0 x1874671476738816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827725.514389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476738816/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827725.514397:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827725.514401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827725.514404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476738816:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827725.514432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827722/real 1787827725] req@ffff9a78841da580 x1874671476739328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827725.514445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476739328/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827725.514455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827725.514460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827725.514463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739328:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827726.537969:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827727.562024:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787827727.562038:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827727.562038:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827727.562050:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827727.562066:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2095 00000100:00080000:2.0:1787827727.562080:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827727.562088:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827727.562156:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476739456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827727.562186:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827727.562190:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827727.562191:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827727.562195:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2095 00000100:00080000:2.0:1787827727.562198:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827727.562201:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827727.562212:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827727.562224:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476739584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827727.562235:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827727.562239:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827727.562240:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827727.562244:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2095 00000100:00080000:2.0:1787827727.562246:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827727.562249:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827727.562255:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827727.562262:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476739712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827727.562278:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476739456:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827727.562281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827727.562292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827727.562296:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827727.562304:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2095 00000100:00080000:2.0:1787827727.562307:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827727.562310:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827727.562314:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827727.562332:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476739840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827727.562339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827727.562341:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827727.562343:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827727.562347:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2095 00000100:00080000:2.0:1787827727.562350:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827727.562355:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827727.562358:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827727.562372:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476739968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827727.562420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827727.562492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827727/real 1787827727] req@ffff9a78841d8b40 x1874671476739456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827727.562524:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476739456/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827727.562555:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827727.562564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827727.562572:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476739456:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827727.562589:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739584:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827727.562631:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827727.562657:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827727/real 1787827727] req@ffff9a78841db840 x1874671476739584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827727.562693:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476739584/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827727.562709:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827727.562715:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827727.562721:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739584:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827727.562763:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739712:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827727.562804:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827727.562828:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827727/real 1787827727] req@ffff9a78841d92c0 x1874671476739712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827727.562901:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476739712/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827727.562917:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827727.562923:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827727.562928:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739712:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827727.562942:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739840:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827727.562980:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827727.563002:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827727/real 1787827727] req@ffff9a78841d9680 x1874671476739840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827727.563024:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476739840/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827727.563055:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827727.563061:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827727.563066:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739840:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827727.563078:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739968:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827727.563117:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827727.563169:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827727/real 1787827727] req@ffff9a78841da940 x1874671476739968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827727.563193:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476739968/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827727.563211:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827727.563216:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827727.563223:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476739968:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827728.586098:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827729.610157:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827730.634071:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827731.658031:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827732.681973:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827732.681982:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827732.681985:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827732.681993:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2100 00000100:00080000:3.0:1787827732.682029:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827732.682035:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827732.682069:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476740096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827732.682107:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827732.682126:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827732.682128:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827732.682131:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2100 00000100:00080000:3.0:1787827732.682134:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827732.682137:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827732.682145:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827732.682154:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476740224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827732.682162:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827732.682164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827732.682166:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827732.682171:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476740096:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827732.682172:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2100 00000100:00080000:3.0:1787827732.682178:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827732.682182:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827732.682185:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827732.682194:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476740352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827732.682200:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827732.682203:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827732.682204:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827732.682207:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2100 00000100:00080000:3.0:1787827732.682222:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827732.682225:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827732.682229:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827732.682233:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827732.682249:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476740480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827732.682263:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827732.682267:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827732.682269:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827732.682274:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827732/real 1787827732] req@ffff9a78a6775680 x1874671476740096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827732.682276:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2100 00000100:00080000:3.0:1787827732.682284:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827732.682290:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827732.682295:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827732.682305:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476740096/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827732.682310:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476740608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827732.682345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827732.682350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827732.682354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476740096:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827732.682375:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740224:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827732.682396:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827732.682406:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827732/real 1787827732] req@ffff9a78a6776d00 x1874671476740224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827732.682416:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476740224/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827732.682422:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827732.682425:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827732.682428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740224:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827732.682435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740352:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787827732.682447:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:0.0:1787827732.682448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827732.682458:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827732/real 1787827732] req@ffff9a78a67761c0 x1874671476740352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827732.682466:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476740352/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827732.682471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827732.682473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827732.682475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740352:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827732.682479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740480:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827732.682489:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827732.682506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827732/real 1787827732] req@ffff9a78a6774780 x1874671476740480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827732.682513:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476740480/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827732.682520:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827732.682522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827732.682524:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740480:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827732.682528:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740608:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827732.682538:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827732.682544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827732/real 1787827732] req@ffff9a78a67770c0 x1874671476740608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827732.682551:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476740608/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827732.682557:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827732.682559:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827732.682561:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740608:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827733.706025:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827734.729983:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827735.754061:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827736.778023:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827737.801997:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827737.802005:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827737.802009:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827737.802018:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2105 00000100:00080000:0.0:1787827737.802026:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827737.802031:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827737.802064:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827737.802078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476740736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827737.802102:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827737.802105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827737.802106:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827737.802109:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2105 00000100:00080000:0.0:1787827737.802112:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827737.802116:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827737.802124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827737.802136:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3e580 x1874671476740864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827737.802137:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476740736:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827737.802152:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827737.802156:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827737.802158:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827737.802162:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2105 00000100:00080000:0.0:1787827737.802167:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827737.802171:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827737.802176:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827737.802190:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476740992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827737.802199:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827737.802208:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827737.802213:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827737.802215:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827737.802221:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2105 00000100:00080000:0.0:1787827737.802224:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827737.802228:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827737.802233:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827737.802245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827737/real 1787827737] req@ffff9a7890a3c3c0 x1874671476740736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827737.802248:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476741120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827737.802275:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827737.802278:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827737.802279:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787827737.802283:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476740736/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827737.802284:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2105 00000100:00080000:0.0:1787827737.802290:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827737.802296:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827737.802303:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787827737.802306:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787827737.802313:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476741248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827737.802313:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827737.802322:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476740736:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827737.802349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740864:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827737.802365:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827737.802374:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827737/real 1787827737] req@ffff9a7890a3e580 x1874671476740864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827737.802382:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3e580 x1874671476740864/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827737.802388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827737.802390:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827737.802392:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740864:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827737.802396:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740992:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827737.802406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827737.802412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827737/real 1787827737] req@ffff9a7890a3c000 x1874671476740992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827737.802419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476740992/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827737.802424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827737.802426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827737.802428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476740992:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827737.802431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741120:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827737.802441:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827737.802447:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827737/real 1787827737] req@ffff9a7890a3f480 x1874671476741120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827737.802454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476741120/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827737.802460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827737.802462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827737.802464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741120:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827737.802467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741248:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827737.802476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827737.802482:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827737/real 1787827737] req@ffff9a7890a3de00 x1874671476741248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827737.802489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476741248/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827737.802495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827737.802497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827737.802500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741248:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827738.826010:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827739.850185:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827740.873947:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827741.898034:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827742.921963:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827742.921972:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827742.921976:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827742.921985:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2110 00000100:00080000:1.0:1787827742.921996:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827742.922002:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827742.922058:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827742.922067:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476741376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827742.922093:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827742.922095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827742.922097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827742.922100:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2110 00000100:00080000:1.0:1787827742.922103:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827742.922115:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827742.922123:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827742.922133:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476741504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827742.922138:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476741376:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827742.922154:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827742.922156:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827742.922157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827742.922162:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2110 00000100:00080000:1.0:1787827742.922163:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827742.922166:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827742.922169:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827742.922178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476741632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827742.922184:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827742.922187:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827742.922188:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827742.922191:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2110 00000100:00080000:1.0:1787827742.922194:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827742.922196:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827742.922201:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827742.922202:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827742.922212:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476741760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827742.922219:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827742.922221:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827742.922222:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827742.922225:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2110 00000100:00080000:1.0:1787827742.922229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827742.922231:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827742.922235:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827742.922245:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476741888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827742.922263:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827742/real 1787827742] req@ffff9a7890b1fc00 x1874671476741376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827742.922296:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476741376/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827742.922306:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827742.922309:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827742.922312:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476741376:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827742.922331:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741504:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827742.922349:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827742.922358:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827742/real 1787827742] req@ffff9a7890b1c000 x1874671476741504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827742.922367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476741504/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827742.922373:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827742.922375:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827742.922377:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741504:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827742.922381:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741632:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827742.922391:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827742.922397:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827742/real 1787827742] req@ffff9a7890b1f0c0 x1874671476741632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827742.922405:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476741632/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827742.922410:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827742.922412:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827742.922414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741632:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827742.922418:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741760:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827742.922428:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827742.922433:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827742/real 1787827742] req@ffff9a7890b1e940 x1874671476741760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827742.922440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476741760/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827742.922446:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827742.922448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827742.922450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741760:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827742.922454:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741888:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827742.922463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827742.922469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827742/real 1787827742] req@ffff9a7890b1da40 x1874671476741888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827742.922476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476741888/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827767 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827742.922482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827742.922484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827742.922486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476741888:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827743.945996:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827744.970083:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827745.994065:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827747.018002:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827748.041938:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827748.041948:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827748.041953:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827748.041963:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2115 00000100:00080000:3.0:1787827748.041974:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827748.041997:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827748.042034:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476742016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827748.042064:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827748.042067:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827748.042069:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827748.042073:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2115 00000100:00080000:3.0:1787827748.042076:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827748.042080:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827748.042090:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827748.042101:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476742144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827748.042112:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827748.042116:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827748.042118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827748.042123:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476742016:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827748.042125:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2115 00000100:00080000:3.0:1787827748.042132:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827748.042137:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827748.042142:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827748.042153:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476742272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827748.042169:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827748.042172:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827748.042174:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827748.042177:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2115 00000100:00080000:3.0:1787827748.042189:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827748.042193:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827748.042198:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787827748.042200:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827748.042214:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476742400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827748.042223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827748.042226:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827748.042228:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827748.042232:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2115 00000100:00080000:3.0:1787827748.042236:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827748.042240:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827748.042246:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827748.042255:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff9a78a6776940 x1874671476742016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827748.042259:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476742528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827748.042299:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476742016/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827748.042313:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827748.042318:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827748.042324:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476742016:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827748.042349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742144:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827748.042373:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:2.0:1787827748.042388:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827748.042392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff9a78a6776580 x1874671476742144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827748.042414:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476742144/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827748.042426:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827748.042431:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827748.042436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742144:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827748.042445:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742272:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827748.042472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827748.042484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff9a78a6774000 x1874671476742272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827748.042495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476742272/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827748.042509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827748.042512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827748.042515:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742272:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827748.042521:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742400:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827748.042557:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827748.042569:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff9a78a67752c0 x1874671476742400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827748.042582:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476742400/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827748.042593:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827748.042613:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827748.042618:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742400:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827748.042626:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742528:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827748.042652:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827748.042669:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827748/real 1787827748] req@ffff9a78a6775a40 x1874671476742528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827748.042687:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476742528/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827748.042698:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827748.042701:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827748.042705:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742528:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827749.066104:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827750.090074:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827751.113991:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827752.137976:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827753.161915:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827753.161928:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827753.161933:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827753.161957:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2120 00000100:00080000:0.0:1787827753.161971:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827753.161972:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827753.161980:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827753.162035:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476742656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827753.162062:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827753.162065:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827753.162067:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827753.162071:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2120 00000100:00080000:0.0:1787827753.162074:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827753.162078:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827753.162089:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827753.162097:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476742656:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827753.162101:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476742784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827753.162121:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827753.162124:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827753.162125:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827753.162128:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2120 00000100:00080000:0.0:1787827753.162131:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827753.162134:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827753.162138:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827753.162149:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476742912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827753.162157:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827753.162160:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827753.162161:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827753.162166:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2120 00000100:00080000:0.0:1787827753.162179:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787827753.162185:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827753.162186:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827753.162197:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827753.162213:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476743040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827753.162223:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827753.162226:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827753.162228:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827753.162232:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2120 00000100:00080000:0.0:1787827753.162236:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827753.162239:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9a7890a3fc00 x1874671476742656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827753.162242:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827753.162249:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827753.162274:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476743168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827753.162280:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476742656/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827753.162291:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827753.162295:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827753.162300:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476742656:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827753.162336:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742784:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827753.162361:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827753.162371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9a7890a3da40 x1874671476742784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827753.162384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476742784/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827753.162412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827753.162416:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827753.162420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742784:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827753.162427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742912:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827753.162448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827753.162467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9a7890a3de00 x1874671476742912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827753.162475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476742912/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827753.162486:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827753.162488:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827753.162490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476742912:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827753.162495:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743040:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827753.162507:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827753.162513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9a7890a3f480 x1874671476743040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827753.162521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476743040/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827753.162539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827753.162541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827753.162543:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743040:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827753.162547:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743168:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827753.162565:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827753.162572:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827753/real 1787827753] req@ffff9a7890a3c000 x1874671476743168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827753.162580:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476743168/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827753.162587:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827753.162600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827753.162603:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743168:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827754.185960:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827755.209928:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827756.233979:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827757.257944:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827758.281977:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827758.281984:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827758.281985:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827758.282008:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827758.282018:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2125 00000100:00080000:1.0:1787827758.282028:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.282034:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827758.282066:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476743296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827758.282087:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827758.282091:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827758.282094:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827758.282098:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2125 00000100:00080000:1.0:1787827758.282118:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.282122:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827758.282139:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827758.282142:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476743296:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827758.282152:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476743424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827758.282163:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827758.282168:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827758.282170:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827758.282173:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2125 00000100:00080000:1.0:1787827758.282192:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.282197:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827758.282201:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787827758.282201:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827758.282213:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476743552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827758.282223:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827758.282226:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827758.282228:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827758.282231:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2125 00000100:00080000:1.0:1787827758.282235:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.282238:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827758.282243:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827758.282255:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476743680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827758.282255:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9a7890b1f840 x1874671476743296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827758.282266:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827758.282269:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827758.282271:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787827758.282274:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476743296/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827758.282275:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2125 00000100:00080000:1.0:1787827758.282278:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827758.282282:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787827758.282285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787827758.282287:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787827758.282290:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827758.282294:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476743296:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827758.282296:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476743808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827758.282318:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743424:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827758.282343:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827758.282356:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9a7890b1f480 x1874671476743424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827758.282365:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476743424/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827758.282379:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827758.282397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827758.282400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743424:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827758.282405:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743552:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827758.282419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827758.282432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9a7890b1e580 x1874671476743552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827758.282440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476743552/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827758.282445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827758.282447:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827758.282449:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743552:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827758.282452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743680:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827758.282463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827758.282469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9a7890b1de00 x1874671476743680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827758.282476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476743680/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827758.282482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827758.282484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827758.282486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743680:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827758.282489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743808:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827758.282499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827758.282504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827758/real 1787827758] req@ffff9a7890b1c3c0 x1874671476743808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827758.282511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476743808/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827758.282517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827758.282519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827758.282521:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476743808:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827759.306082:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827760.329945:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827761.354041:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827762.378057:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827763.401994:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827763.402004:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827763.402009:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827763.402021:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827763.402021:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2130 00000100:00080000:2.0:1787827763.402035:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827763.402043:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827763.402088:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476743936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827763.402113:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827763.402116:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827763.402118:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827763.402121:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2130 00000100:00080000:2.0:1787827763.402123:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827763.402126:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827763.402136:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827763.402145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476744064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827763.402151:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476743936:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827763.402155:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827763.402161:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827763.402162:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827763.402165:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2130 00000100:00080000:2.0:1787827763.402167:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827763.402169:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827763.402172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827763.402181:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476744192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827763.402188:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827763.402190:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827763.402191:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827763.402194:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2130 00000100:00080000:2.0:1787827763.402215:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827763.402218:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827763.402222:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827763.402226:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827763.402234:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476744320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827763.402242:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827763.402244:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827763.402246:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827763.402249:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2130 00000100:00080000:2.0:1787827763.402252:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827763.402254:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827763.402257:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827763.402266:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476744448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827763.402286:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9a78841da940 x1874671476743936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827763.402302:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476743936/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827763.402311:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827763.402314:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827763.402317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476743936:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827763.402339:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744064:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827763.402354:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827763.402362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9a78841d9680 x1874671476744064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827763.402370:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476744064/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827763.402377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827763.402379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827763.402381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744064:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827763.402386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744192:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827763.402400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827763.402411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9a78841d92c0 x1874671476744192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827763.402424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476744192/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827763.402432:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827763.402436:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827763.402440:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744192:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827763.402447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744320:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827763.402466:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827763.402474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9a78841db840 x1874671476744320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827763.402482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476744320/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827763.402489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827763.402492:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827763.402494:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744320:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827763.402498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744448:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827763.402508:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827763.402514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9a78841d8b40 x1874671476744448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827763.402522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476744448/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827763.402529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827763.402531:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827763.402532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744448:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827764.426007:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827765.450072:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827766.474097:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827767.498052:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827768.521956:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827768.521970:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827768.521976:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827768.521979:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827768.521991:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2136 00000100:00080000:1.0:1787827768.522004:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827768.522013:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827768.522065:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476744576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827768.522094:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827768.522099:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827768.522102:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827768.522107:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2136 00000100:00080000:1.0:1787827768.522112:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827768.522117:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827768.522128:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476744576:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787827768.522129:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827768.522146:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1cf00 x1874671476744704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827768.522155:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827768.522158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827768.522159:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827768.522162:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2136 00000100:00080000:1.0:1787827768.522166:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827768.522169:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827768.522173:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827768.522183:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476744832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827768.522189:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827768.522192:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827768.522194:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827768.522197:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2136 00000100:00080000:1.0:1787827768.522199:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827768.522202:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827768.522206:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827768.522207:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827768.522220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476744960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827768.522231:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827768.522235:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827768.522237:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827768.522242:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2136 00000100:00080000:1.0:1787827768.522246:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827768.522251:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827768.522257:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827768.522262:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9a7890b1d680 x1874671476744576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827768.522273:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476745088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827768.522281:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476744576/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827768.522314:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827768.522318:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827768.522321:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476744576:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827768.522346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744704:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827768.522362:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827768.522371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9a7890b1cf00 x1874671476744704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827768.522379:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1cf00 x1874671476744704/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827768.522390:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827768.522392:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827768.522394:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744704:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827768.522398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744832:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827768.522407:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827768.522413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9a7890b1c3c0 x1874671476744832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827768.522420:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476744832/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827768.522425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827768.522427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827768.522429:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744832:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827768.522433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744960:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827768.522443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827768.522448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9a7890b1de00 x1874671476744960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827768.522456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476744960/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827768.522462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827768.522464:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827768.522466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476744960:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827768.522470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745088:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827768.522480:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827768.522486:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9a7890b1e580 x1874671476745088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827768.522493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476745088/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827768.522500:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827768.522501:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827768.522503:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745088:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827769.545961:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827770.570051:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827771.594010:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827772.618119:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827773.641933:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827773.641944:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827773.641948:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827773.641959:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2141 00000100:00080000:3.0:1787827773.641971:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827773.641978:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827773.642015:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476745216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827773.642058:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827773.642063:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827773.642065:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827773.642070:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2141 00000100:00080000:3.0:1787827773.642074:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827773.642079:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827773.642090:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827773.642104:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476745216:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827773.642106:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476745344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827773.642130:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827773.642134:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827773.642136:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827773.642140:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2141 00000100:00080000:3.0:1787827773.642144:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827773.642148:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827773.642153:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827773.642166:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476745472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827773.642173:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827773.642181:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827773.642185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827773.642188:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827773.642193:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2141 00000100:00080000:3.0:1787827773.642197:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827773.642201:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827773.642214:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827773.642222:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9a78a6777480 x1874671476745216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827773.642232:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476745600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827773.642241:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476745216/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827773.642246:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827773.642254:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827773.642255:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827773.642257:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827773.642260:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787827773.642266:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2141 00000100:00100000:0.0:1787827773.642267:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476745216:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827773.642273:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827773.642280:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827773.642285:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827773.642291:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745344:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827773.642299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476745728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827773.642314:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827773.642327:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9a78a6774b40 x1874671476745344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827773.642337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476745344/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827773.642344:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827773.642347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827773.642349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745344:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827773.642355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745472:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827773.642367:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827773.642375:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9a78a6775a40 x1874671476745472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827773.642383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476745472/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827773.642389:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827773.642391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827773.642393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745472:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827773.642399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745600:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827773.642411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827773.642423:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9a78a67752c0 x1874671476745600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827773.642431:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476745600/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787827773.642437:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827773.642440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827773.642447:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827773.642450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745600:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827773.642466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745728:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827773.642477:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827773.642483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9a78a6774000 x1874671476745728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827773.642495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476745728/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827773.642501:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827773.642503:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827773.642505:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745728:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827774.666146:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827775.689995:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827776.714024:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827777.737988:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827778.761944:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.761952:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827778.761955:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.761963:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2146 00000100:00080000:3.0:1787827778.761972:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.761979:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827778.762013:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476745856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827778.762032:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.762035:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827778.762037:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.762039:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2146 00000100:00080000:3.0:1787827778.762042:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.762044:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827778.762051:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827778.762059:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476745984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827778.762066:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.762069:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827778.762070:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.762073:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2146 00000100:00080000:3.0:1787827778.762075:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.762078:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827778.762083:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476745856:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787827778.762083:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827778.762094:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476746112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827778.762100:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.762102:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827778.762104:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.762106:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2146 00000100:00080000:3.0:1787827778.762108:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.762110:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827778.762113:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827778.762121:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476746240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827778.762128:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.762130:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827778.762132:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.762134:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2146 00000100:00080000:3.0:1787827778.762137:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.762142:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827778.762143:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787827778.762147:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827778.762157:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476746368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827778.762198:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9a78a6776580 x1874671476745856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827778.762213:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476745856/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827778.762222:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827778.762225:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827778.762229:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476745856:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787827778.762234:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827778.762235:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745984:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827778.762254:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827778.762262:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9a78a6776940 x1874671476745984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827778.762270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476745984/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827778.762279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827778.762282:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827778.762284:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476745984:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827778.762302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746112:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827778.762319:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827778.762326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9a78a67770c0 x1874671476746112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827778.762333:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476746112/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827778.762338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827778.762340:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827778.762342:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746112:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827778.762346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746240:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827778.762355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827778.762361:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9a78a6774780 x1874671476746240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827778.762368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476746240/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827778.762374:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827778.762376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827778.762378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746240:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827778.762382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746368:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827778.762391:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827778.762396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9a78a67761c0 x1874671476746368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827778.762403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476746368/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827778.762409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827778.762410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827778.762412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746368:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827779.785948:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827780.810085:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827781.834011:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827782.858038:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827783.881901:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.881909:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827783.881912:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827783.881920:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2151 00000100:00080000:3.0:1787827783.881929:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.881935:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827783.881966:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476746496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827783.881988:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.881991:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827783.881992:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827783.881995:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2151 00000100:00080000:3.0:1787827783.881997:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.882000:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827783.882008:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827783.882017:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476746624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827783.882023:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476746496:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827783.882028:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.882033:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827783.882034:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827783.882037:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2151 00000100:00080000:3.0:1787827783.882039:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.882041:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827783.882044:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827783.882053:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476746752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827783.882059:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.882062:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827783.882063:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827783.882066:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2151 00000100:00080000:3.0:1787827783.882076:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.882080:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827783.882084:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787827783.882086:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827783.882097:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476746880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827783.882103:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.882105:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827783.882107:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827783.882109:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2151 00000100:00080000:3.0:1787827783.882112:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.882115:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827783.882119:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827783.882126:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9a78a6776d00 x1874671476746496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827783.882129:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476747008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827783.882164:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476746496/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827783.882174:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827783.882177:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827783.882181:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476746496:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827783.882202:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746624:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827783.882218:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:2.0:1787827783.882225:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827783.882230:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9a78a6775680 x1874671476746624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827783.882240:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476746624/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827783.882246:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827783.882248:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827783.882251:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746624:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827783.882254:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746752:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827783.882263:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827783.882268:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9a78a6777840 x1874671476746752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827783.882276:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476746752/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827783.882280:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827783.882282:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827783.882284:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746752:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827783.882287:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746880:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827783.882296:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827783.882301:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9a78a6774f00 x1874671476746880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827783.882308:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476746880/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827783.882315:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827783.882317:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827783.882319:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476746880:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827783.882322:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747008:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827783.882331:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827783.882336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9a78a67743c0 x1874671476747008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827783.882342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476747008/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827783.882349:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827783.882350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827783.882352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747008:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827784.906019:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827785.929999:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827786.954125:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827787.978139:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827789.001949:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827789.001962:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827789.001967:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827789.001979:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2156 00000100:00080000:1.0:1787827789.001994:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827789.002002:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827789.002039:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827789.002067:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476747136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827789.002103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827789.002107:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827789.002108:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827789.002112:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2156 00000100:00080000:1.0:1787827789.002119:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827789.002123:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827789.002134:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827789.002147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476747264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827789.002155:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827789.002158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827789.002160:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827789.002163:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2156 00000100:00080000:1.0:1787827789.002165:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827789.002167:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827789.002172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827789.002175:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476747136:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827789.002182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476747392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827789.002191:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827789.002194:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827789.002196:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827789.002202:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2156 00000100:00080000:1.0:1787827789.002206:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827789.002210:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827789.002217:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827789.002231:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476747520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827789.002242:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827789.002247:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827789.002249:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827789.002252:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2156 00000100:00080000:1.0:1787827789.002256:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827789.002259:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827789.002264:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827789.002274:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476747648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827789.002305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747264:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827789.002342:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747392:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827789.002372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747520:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827789.002401:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747648:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827790.026187:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827791.050031:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827792.074184:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827792.074190:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787827792.074238:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827792.074257:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827792.074298:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827792.074307:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827792.074318:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827792.074329:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787827792.074342:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827792] req@ffff9a7890b1da40 x1874671476747392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827792.074372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476747392/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827792.074385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827792.074391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827792.074396:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747392:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827792.074405:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827792] req@ffff9a7890b1e940 x1874671476747520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827792.074419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476747520/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827792.074430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827792.074434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827792.074437:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747520:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827792.074444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827792] req@ffff9a7890b1f0c0 x1874671476747648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827792.074455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476747648/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827792.074491:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827792.074496:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827792.074500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747648:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827792.074507:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827792] req@ffff9a7890b1f480 x1874671476747136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827792.074522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476747136/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827792.074529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827792.074533:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827792.074536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476747136:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827792.074543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827789/real 1787827792] req@ffff9a7890b1f840 x1874671476747264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827792.074556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476747264/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827792.074564:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827792.074567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827792.074571:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747264:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787827793.098016:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827794.121996:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.122004:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827794.122008:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.122018:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2161 00000100:00080000:1.0:1787827794.122027:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827794.122050:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827794.122082:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476747776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827794.122110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.122114:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827794.122116:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.122121:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2161 00000100:00080000:1.0:1787827794.122125:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827794.122128:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827794.122141:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827794.122143:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827794.122151:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476747776:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827794.122152:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476747904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827794.122171:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.122174:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827794.122176:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.122180:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2161 00000100:00080000:1.0:1787827794.122182:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827794.122185:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827794.122188:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827794.122198:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476748032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827794.122210:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.122215:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787827794.122216:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827794.122217:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.122225:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2161 00000100:00080000:1.0:1787827794.122228:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827794.122231:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827794.122235:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827794.122260:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476748160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827794.122267:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827794.122269:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827794.122271:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827794.122275:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2161 00000100:00080000:1.0:1787827794.122278:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827794.122282:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9a7890b1c000 x1874671476747776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827794.122283:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827794.122290:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827794.122304:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476748288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827794.122308:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476747776/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827794.122323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827794.122326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827794.122331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476747776:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827794.122355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747904:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827794.122382:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827794.122397:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9a7890b1fc00 x1874671476747904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827794.122419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476747904/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827794.122427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827794.122430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827794.122434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476747904:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827794.122441:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748032:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827794.122460:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827794.122472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9a7890b1f840 x1874671476748032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827794.122484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476748032/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827794.122499:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827794.122503:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827794.122506:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748032:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827794.122513:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748160:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827794.122533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827794.122546:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9a7890b1f480 x1874671476748160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827794.122560:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476748160/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827794.122572:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827794.122576:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827794.122579:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748160:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827794.122585:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748288:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827794.122604:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827794.122617:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827794/real 1787827794] req@ffff9a7890b1f0c0 x1874671476748288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827794.122630:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476748288/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827794.122640:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827794.122643:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827794.122647:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748288:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827795.146122:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827796.170039:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827797.194063:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827798.218037:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827799.241988:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827799.242005:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827799.242011:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827799.242028:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2166 00000100:00080000:2.0:1787827799.242046:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827799.242055:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827799.242085:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827799.242121:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476748416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827799.242157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827799.242162:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827799.242164:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827799.242170:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2166 00000100:00080000:2.0:1787827799.242175:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827799.242181:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827799.242196:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827799.242216:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476748544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827799.242234:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827799.242236:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476748416:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827799.242241:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827799.242250:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827799.242261:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2166 00000100:00080000:2.0:1787827799.242265:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827799.242284:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827799.242293:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827799.242311:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476748672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827799.242321:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827799.242324:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827799.242326:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827799.242330:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2166 00000100:00080000:2.0:1787827799.242349:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827799.242356:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827799.242363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787827799.242366:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827799.242383:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476748800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827799.242394:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827799.242398:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827799.242401:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827799.242404:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2166 00000100:00080000:2.0:1787827799.242407:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827799.242417:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827799.242424:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827799.242438:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476748928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827799.242491:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9a78841da580 x1874671476748416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827799.242534:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476748416/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827799.242550:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827799.242557:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827799.242563:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476748416:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827799.242596:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748544:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827799.242653:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827799.242685:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9a78841d83c0 x1874671476748544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827799.242705:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476748544/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827799.242718:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827799.242725:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827799.242731:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748544:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827799.242741:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748672:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827799.242789:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827799.242824:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9a78841d8000 x1874671476748672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827799.242840:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476748672/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827799.242880:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827799.242885:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827799.242890:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748672:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827799.242899:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748800:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827799.242928:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827799.242945:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9a78841db480 x1874671476748800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827799.242963:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476748800/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827799.242994:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827799.242999:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827799.243004:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748800:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827799.243015:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748928:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827799.243043:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827799.243077:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9a78841d9e00 x1874671476748928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827799.243092:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476748928/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827799.243105:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827799.243111:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827799.243115:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476748928:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827800.266037:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827801.290050:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827802.314049:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827803.338080:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827804.361950:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827804.361960:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827804.361966:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827804.361978:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2171 00000100:00080000:2.0:1787827804.361991:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827804.362001:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787827804.362013:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827804.362067:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476749056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827804.362100:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827804.362105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827804.362106:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827804.362111:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2171 00000100:00080000:2.0:1787827804.362117:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827804.362122:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827804.362135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827804.362143:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476749056:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827804.362156:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476749184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827804.362174:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827804.362178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827804.362180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827804.362186:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2171 00000100:00080000:2.0:1787827804.362206:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827804.362212:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827804.362221:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787827804.362226:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827804.362236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476749312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827804.362246:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827804.362249:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827804.362251:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827804.362255:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2171 00000100:00080000:2.0:1787827804.362258:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827804.362262:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827804.362267:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787827804.362274:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476749440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827804.362287:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827804.362293:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787827804.362294:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9a78841dbc00 x1874671476749056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827804.362296:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827804.362307:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2171 00000100:00080000:2.0:1787827804.362316:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827804.362330:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827804.362336:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827804.362342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476749056/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827804.362352:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476749568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827804.362359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827804.362369:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827804.362374:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476749056:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827804.362403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749184:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827804.362430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827804.362445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9a78841da1c0 x1874671476749184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827804.362454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476749184/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827804.362460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827804.362462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827804.362465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749184:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827804.362470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749312:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827804.362485:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827804.362492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9a78841db0c0 x1874671476749312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827804.362499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476749312/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827804.362504:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827804.362512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827804.362514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749312:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827804.362518:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749440:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827804.362530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827804.362536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9a78841d9a40 x1874671476749440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827804.362543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476749440/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827804.362550:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827804.362551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827804.362553:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749440:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827804.362557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749568:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827804.362567:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827804.362573:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9a78841d8780 x1874671476749568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827804.362581:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476749568/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827804.362586:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827804.362588:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827804.362607:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749568:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827805.386108:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827806.410183:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827807.434091:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827808.458069:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827809.482044:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827809.482057:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827809.482064:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827809.482077:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2176 00000100:00080000:0.0:1787827809.482091:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827809.482101:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827809.482109:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827809.482153:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3e580 x1874671476749696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827809.482191:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827809.482197:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827809.482199:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827809.482204:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2176 00000100:00080000:0.0:1787827809.482208:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827809.482214:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827809.482229:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827809.482245:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476749824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827809.482254:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476749696:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827809.482276:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827809.482280:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827809.482282:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827809.482288:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2176 00000100:00080000:0.0:1787827809.482293:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827809.482297:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827809.482303:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827809.482321:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476749952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827809.482334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827809.482338:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827809.482340:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827809.482345:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2176 00000800:00000100:3.0:1787827809.482352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827809.482353:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827809.482366:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827809.482375:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827809.482391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476750080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827809.482404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827809.482409:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827809.482413:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827809.482417:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2176 00000100:00080000:0.0:1787827809.482422:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827809.482428:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827809.482435:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827809.482439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9a7890a3e580 x1874671476749696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827809.482454:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476750208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827809.482480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3e580 x1874671476749696/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827809.482498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827809.482506:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827809.482513:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476749696:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827809.482550:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749824:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827809.482615:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827809.482643:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9a7890a3c3c0 x1874671476749824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827809.482669:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476749824/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827809.482699:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827809.482707:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827809.482715:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749824:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827809.482724:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749952:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827809.482754:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827809.482770:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9a7890a3c000 x1874671476749952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827809.482801:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476749952/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827809.482811:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827809.482815:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827809.482819:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476749952:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827809.482826:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750080:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827809.482879:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827809.482894:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9a7890a3f480 x1874671476750080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827809.482908:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476750080/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827809.482919:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827809.482923:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827809.482928:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750080:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827809.482936:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750208:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827809.482954:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827809.482965:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9a7890a3de00 x1874671476750208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827809.482978:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476750208/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827809.482989:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827809.482993:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827809.482996:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750208:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827810.505984:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827811.529989:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827812.554000:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827813.578068:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827814.601969:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827814.601987:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827814.601994:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.602010:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2182 00000100:00080000:0.0:1787827814.602026:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827814.602039:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827814.602082:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827814.602097:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476750336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827814.602145:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827814.602151:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827814.602154:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.602159:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2182 00000100:00080000:0.0:1787827814.602165:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827814.602172:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827814.602185:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827814.602209:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476750336:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827814.602211:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476750464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827814.602238:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827814.602242:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827814.602244:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.602250:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2182 00000100:00080000:0.0:1787827814.602255:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827814.602272:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827814.602279:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827814.602295:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476750592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827814.602311:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827814.602312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827814.602323:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827814.602326:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.602334:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2182 00000100:00080000:0.0:1787827814.602340:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827814.602346:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827814.602354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827814.602370:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476750720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827814.602384:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827814.602392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827814] req@ffff9a7890a3da40 x1874671476750336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827814.602393:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827814.602399:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.602414:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2182 00000100:00080000:0.0:1787827814.602424:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827814.602432:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827814.602439:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827814.602447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476750336/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827814.602459:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476750848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827814.602474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827814.602487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827814.602494:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476750336:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827814.602552:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750464:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827814.602586:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827814.602605:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827814] req@ffff9a7890a3fc00 x1874671476750464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827814.602631:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476750464/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827814.602648:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827814.602654:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827814.602661:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750464:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827814.602673:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750592:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827814.602733:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827814.602754:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827814] req@ffff9a7890a3de00 x1874671476750592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827814.602770:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476750592/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827814.602781:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827814.602785:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827814.602789:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750592:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827814.602796:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750720:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827814.602821:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827814.602878:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827814] req@ffff9a7890a3f480 x1874671476750720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827814.602905:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476750720/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827814.602926:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827814.602933:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827814.602939:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750720:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827814.602951:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750848:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827814.602982:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827814.602999:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827814] req@ffff9a7890a3c000 x1874671476750848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827814.603014:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476750848/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827814.603027:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827814.603030:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827814.603034:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476750848:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827815.626052:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827816.650077:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827817.674012:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827818.697992:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827819.721944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.721962:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827819.721967:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.721979:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2187 00000100:00080000:3.0:1787827819.721992:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.722001:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827819.722045:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476750976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827819.722087:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.722095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827819.722100:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.722107:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2187 00000100:00080000:3.0:1787827819.722113:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.722121:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827819.722136:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827819.722146:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476750976:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827819.722157:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476751104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827819.722176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.722181:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827819.722184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.722191:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2187 00000100:00080000:3.0:1787827819.722208:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.722215:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827819.722222:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827819.722241:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476751232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827819.722254:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827819.722259:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.722265:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827819.722268:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.722274:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2187 00000100:00080000:3.0:1787827819.722280:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.722285:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827819.722294:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827819.722310:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476751360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827819.722316:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9a78a6775e00 x1874671476750976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827819.722328:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.722334:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827819.722337:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.722344:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2187 00000100:00100000:0.0:1787827819.722345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476750976/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827819.722349:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.722356:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787827819.722364:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:3.0:1787827819.722364:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787827819.722372:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827819.722379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476750976:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827819.722380:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476751488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827819.722418:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751104:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827819.722457:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827819.722482:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9a78a6777c00 x1874671476751104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827819.722539:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476751104/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827819.722559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827819.722576:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00010000:00010000:2.0:1787827819.722579:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827819.722583:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751104:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827819.722597:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751232:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827819.722629:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827819.722645:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9a78a67743c0 x1874671476751232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827819.722660:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476751232/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827819.722688:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827819.722693:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827819.722697:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751232:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827819.722704:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751360:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827819.722727:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827819.722739:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9a78a6774f00 x1874671476751360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827819.722765:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476751360/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827819.722787:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827819.722791:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827819.722795:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751360:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827819.722802:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751488:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827819.722823:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827819.722885:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9a78a6777840 x1874671476751488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827819.722900:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476751488/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827819.722911:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827819.722915:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827819.722919:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751488:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827820.745930:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827821.769933:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827822.794045:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827823.818052:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827824.842069:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827824.842088:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827824.842097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827824.842112:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2192 00010000:00010000:0.0:1787827824.842123:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827824.842129:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827824.842145:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827824.842194:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476751616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827824.842241:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827824.842246:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827824.842248:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827824.842253:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2192 00000100:00080000:1.0:1787827824.842257:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827824.842261:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827824.842274:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827824.842289:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476751744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827824.842306:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827824.842323:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827824.842326:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827824.842338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476751616:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827824.842341:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2192 00000100:00080000:1.0:1787827824.842357:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827824.842365:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827824.842372:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827824.842395:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476751872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827824.842407:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827824.842413:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827824.842416:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827824.842423:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2192 00000100:00080000:1.0:1787827824.842488:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827824.842495:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827824.842503:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827824.842517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827824.842531:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476752000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827824.842546:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827824.842552:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827824.842555:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827824.842561:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2192 00000100:00080000:1.0:1787827824.842567:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827824.842574:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827824.842581:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827824.842593:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476752128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827824.842652:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9a7890b1e940 x1874671476751616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827824.842681:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476751616/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827824.842697:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827824.842703:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827824.842725:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476751616:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827824.842767:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751744:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827824.842804:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827824.842823:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9a7890b1da40 x1874671476751744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827824.842839:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476751744/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827824.842918:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827824.842923:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827824.842929:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751744:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827824.842938:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751872:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827824.842965:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827824.842980:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9a7890b1e580 x1874671476751872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827824.842996:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476751872/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827824.843006:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827824.843010:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827824.843014:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476751872:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827824.843022:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752000:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827824.843044:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827824.843054:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9a7890b1de00 x1874671476752000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827824.843068:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476752000/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827824.843083:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827824.843087:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827824.843091:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752000:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827824.843098:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752128:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827824.843117:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827824.843128:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9a7890b1c3c0 x1874671476752128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827824.843142:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476752128/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827824.843154:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827824.843158:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827824.843161:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752128:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827825.866064:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827826.890102:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827827.914075:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827828.938087:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827829.961993:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827829.962008:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827829.962015:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827829.962030:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2197 00000100:00080000:0.0:1787827829.962045:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827829.962054:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827829.962056:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827829.962112:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476752256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827829.962158:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827829.962162:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827829.962165:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827829.962169:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2197 00000100:00080000:0.0:1787827829.962173:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827829.962177:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827829.962190:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827829.962204:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3e580 x1874671476752384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827829.962211:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476752256:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827829.962221:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827829.962226:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827829.962228:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827829.962233:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2197 00000100:00080000:0.0:1787827829.962236:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827829.962241:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827829.962246:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827829.962261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476752512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827829.962271:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827829.962275:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827829.962277:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827829.962281:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2197 00000100:00080000:0.0:1787827829.962295:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827829.962299:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827829.962305:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827829.962307:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827829.962322:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476752640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827829.962334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827829.962337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827829.962339:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827829.962344:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2197 00000100:00080000:0.0:1787827829.962347:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827829.962351:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827829.962356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827829.962371:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476752768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827829.962393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9a7890a3c3c0 x1874671476752256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827829.962447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476752256/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827829.962480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827829.962485:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827829.962490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476752256:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827829.962522:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752384:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827829.962551:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827829.962567:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9a7890a3e580 x1874671476752384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827829.962581:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3e580 x1874671476752384/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827829.962590:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827829.962594:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827829.962609:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752384:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827829.962615:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752512:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827829.962631:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827829.962640:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9a7890a3c000 x1874671476752512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827829.962653:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476752512/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827829.962661:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827829.962664:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827829.962667:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752512:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827829.962672:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752640:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827829.962689:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827829.962698:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9a7890a3f480 x1874671476752640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827829.962718:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476752640/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827829.962729:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827829.962732:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827829.962735:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752640:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827829.962741:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752768:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827829.962756:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827829.962765:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9a7890a3de00 x1874671476752768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827829.962777:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476752768/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827829.962806:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827829.962808:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827829.962811:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476752768:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827830.985994:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827832.010097:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827833.034122:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827834.058001:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827835.081926:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.081934:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827835.081938:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.081947:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2202 00000100:00080000:3.0:1787827835.081957:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827835.081966:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827835.082006:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476752896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827835.082038:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.082042:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827835.082045:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.082050:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2202 00000100:00080000:3.0:1787827835.082054:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827835.082060:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827835.082068:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476752896:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787827835.082068:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827835.082084:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476753024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827835.082093:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.082095:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827835.082097:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.082099:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2202 00000100:00080000:3.0:1787827835.082102:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827835.082104:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827835.082107:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827835.082115:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476753152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827835.082122:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.082124:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827835.082125:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.082128:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2202 00000100:00080000:3.0:1787827835.082131:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827835.082136:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827835.082138:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787827835.082140:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827835.082150:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476753280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827835.082157:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827835.082159:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827835.082160:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827835.082163:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2202 00000100:00080000:3.0:1787827835.082165:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827835.082168:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827835.082171:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827835.082180:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476753408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827835.082187:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff9a78a6775680 x1874671476752896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827835.082215:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476752896/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827835.082223:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827835.082227:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827835.082230:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476752896:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827835.082256:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753024:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827835.082295:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:2.0:1787827835.082309:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827835.082315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff9a78a6776d00 x1874671476753024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827835.082329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476753024/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827835.082339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827835.082342:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827835.082344:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753024:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827835.082349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753152:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827835.082362:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827835.082368:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff9a78a67761c0 x1874671476753152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827835.082388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476753152/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827835.082395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827835.082397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827835.082399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753152:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827835.082404:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753280:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827835.082430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827835.082437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff9a78a6774780 x1874671476753280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827835.082445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476753280/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827835.082451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827835.082453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827835.082455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753280:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827835.082458:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753408:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827835.082467:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827835.082472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827835/real 1787827835] req@ffff9a78a67770c0 x1874671476753408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827835.082488:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476753408/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827835.082495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827835.082496:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827835.082498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753408:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827836.106067:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827837.130007:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827838.154040:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827839.178077:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827840.201970:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827840.201980:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827840.201986:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827840.202000:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2207 00000100:00080000:0.0:1787827840.202014:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827840.202026:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827840.202036:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827840.202078:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476753536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827840.202119:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827840.202123:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827840.202124:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827840.202128:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2207 00000100:00080000:0.0:1787827840.202132:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827840.202137:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827840.202149:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827840.202177:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476753664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827840.202195:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827840.202196:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476753536:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827840.202200:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827840.202205:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827840.202215:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2207 00000100:00080000:0.0:1787827840.202219:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827840.202222:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827840.202227:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827840.202241:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476753792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827840.202248:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827840.202252:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827840.202254:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827840.202258:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2207 00000100:00080000:0.0:1787827840.202272:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827840.202275:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827840.202280:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827840.202286:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827840.202291:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476753920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827840.202300:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827840.202303:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827840.202304:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827840.202308:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2207 00000100:00080000:0.0:1787827840.202312:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827840.202314:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827840.202317:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827840.202329:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476754048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827840.202376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9a7890a3fc00 x1874671476753536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827840.202417:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476753536/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827840.202428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827840.202432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827840.202436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476753536:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827840.202472:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753664:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827840.202498:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827840.202512:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9a7890a3da40 x1874671476753664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827840.202525:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476753664/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827840.202534:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827840.202537:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827840.202541:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753664:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827840.202548:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753792:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827840.202568:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827840.202580:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9a7890a3de00 x1874671476753792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827840.202593:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476753792/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827840.202601:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827840.202605:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827840.202608:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753792:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827840.202615:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753920:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827840.202643:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827840.202655:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9a7890a3f480 x1874671476753920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827840.202668:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476753920/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827840.202679:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827840.202682:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827840.202686:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476753920:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827840.202692:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754048:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827840.202720:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827840.202732:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9a7890a3c000 x1874671476754048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827840.202744:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476754048/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827840.202754:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827840.202757:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827840.202761:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754048:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827841.225975:0:47598:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827842.250016:0:47597:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827843.274065:0:14467:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 124/124 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787827843.274130:0:47598:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 124 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787827843.274145:0:47598:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000863a79e/0xebdcd48a4298e0fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400020000 nid: local remote: 0x4059aa0aef8da261 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787827843.274252:0:14467:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 121/121 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787827843.274303:0:47597:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 121 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787827843.274358:0:14467:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 85/85 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787827843.274451:0:14467:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 81/166 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1787827843.274649:0:48516:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 85 locks of 81 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787827843.274670:0:47597:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000009f81abd8/0xebdcd48a4298e8d3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400020000 nid: local remote: 0x4059aa0aef8dbca8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787827843.274919:0:48517:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 81 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787827843.274954:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504400/0xebdcd48a4298e4ab lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8db1cd expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827843.275001:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890939200/0xebdcd48a4298d9d7 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8d8fc2 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827843.275016:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827843.275044:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x52:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275059:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 2/0,0 mode: CR/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787827843.275071:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x51:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275079:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827843.275093:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275103:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x4f:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275112:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275124:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x4e:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275131:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2e00/0xebdcd48a4298e411 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8daa79 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275142:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x4d:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275150:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2200/0xebdcd48a4298e3ee lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8daa1e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275159:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x4c:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275167:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2c00/0xebdcd48a4298e3cb lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da9c3 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275175:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x4b:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275181:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2800/0xebdcd48a4298e3a8 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da968 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787827843.275194:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1787827843.275195:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x4a:0x0]: losing layout lock 00010000:00010000:3.0:1787827843.275201:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827843.275208:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827843.275210:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1c00/0xebdcd48a4298e385 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da90d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275225:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x49:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275227:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827843.275233:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880030c00/0xebdcd48a4298e362 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da8b2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275261:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x48:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275268:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b13600/0xebdcd48a4298e33f lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da857 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275270:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x51:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275288:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x47:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275314:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 2/0,0 mode: CR/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827843.275317:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12600/0xebdcd48a4298e31c lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da7fc expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275345:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x50:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275348:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x46:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275362:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979600/0xebdcd48a4298e2f9 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da7a1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.275369:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787827843.275370:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x45:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275378:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031a00/0xebdcd48a4298e2d6 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da746 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275391:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x44:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275392:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.275399:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887978000/0xebdcd48a4298e2b3 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da6eb expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275405:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x4e:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275437:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275440:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x43:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275462:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979000/0xebdcd48a4298e290 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x42:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da690 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275464:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x4d:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275489:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x42:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275491:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.275502:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148ec00/0xebdcd48a4298e26d lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da635 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275514:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x4c:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275519:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x41:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275536:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.275538:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1200/0xebdcd48a4298e24a lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da5da expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275605:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x40:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.275605:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x4b:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275622:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.275623:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0a00/0xebdcd48a4298e227 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da57f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275656:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x4a:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275658:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x3f:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275669:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.275671:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1600/0xebdcd48a4298e204 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da524 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275698:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x49:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275699:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x3e:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275711:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.275712:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0800/0xebdcd48a4298e1e1 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da4c9 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275741:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x48:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275743:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x3d:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275759:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979a00/0xebdcd48a4298e1be lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da46e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.275759:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275796:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x3c:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.275796:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x47:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275805:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887978c00/0xebdcd48a4298e19b lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da413 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.275811:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275813:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x3b:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275826:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979c00/0xebdcd48a4298e178 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x3a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da3b8 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275829:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x46:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275842:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18600/0xebdcd48a4298edf7 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dca15 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275843:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x3a:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275878:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979400/0xebdcd48a4298e155 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da35d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.275920:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x39:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.275929:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031800/0xebdcd48a4298e132 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da302 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275931:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x45:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275951:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x38:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275955:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18a00/0xebdcd48a4298edd4 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc9ba expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.275980:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1e00/0xebdcd48a4298e10f lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da2a7 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.275985:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x44:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.275993:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x37:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.275997:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18200/0xebdcd48a4298edb1 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc95f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276003:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0400/0xebdcd48a4298e0ec lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da24c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276009:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x43:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276016:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x36:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276022:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ed8e lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x42:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc904 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276027:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0000/0xebdcd48a4298e0c9 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da1f1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276052:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x42:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276059:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x35:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276081:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19c00/0xebdcd48a4298ed6b lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc8a9 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276092:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148fa00/0xebdcd48a4298e0a6 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da196 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276099:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x41:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276110:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x34:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276116:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19200/0xebdcd48a4298ed48 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc84e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276126:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470800/0xebdcd48a4298e083 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da13b expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276132:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x40:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276152:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x33:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276153:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276170:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890855400/0xebdcd48a4298e060 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da0e0 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276173:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x3f:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276192:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276194:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x32:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276215:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148f800/0xebdcd48a4298e03d lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da085 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276217:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x3e:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276235:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276239:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x31:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.276258:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x3d:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276259:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148ee00/0xebdcd48a4298e01a lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8da02a expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.276272:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18c00/0xebdcd48a4298ecbc lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc6e2 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276276:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x30:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276308:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e600/0xebdcd48a4298dff7 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9fcf expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276314:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x3c:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276331:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x2f:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276340:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2e00/0xebdcd48a4298ec99 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc687 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276342:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e400/0xebdcd48a4298dfd4 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9f74 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276394:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x2e:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.276394:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x3b:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276414:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e000/0xebdcd48a4298dfb1 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9f19 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.276414:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2800/0xebdcd48a4298ec76 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc62c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276447:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x3a:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276448:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x2d:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276458:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276461:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd200/0xebdcd48a4298df8e lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9ebe expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276491:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x2c:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.276496:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x39:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276497:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471a00/0xebdcd48a4298df6b lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9e63 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276510:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x2b:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276512:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21dbe00/0xebdcd48a4298ec30 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc576 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276518:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471e00/0xebdcd48a4298df48 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9e08 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276526:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x38:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276530:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x2a:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276538:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da200/0xebdcd48a4298ec0d lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc51b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276541:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031000/0xebdcd48a4298df25 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9dad expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276561:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x37:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276562:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x29:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276576:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031200/0xebdcd48a4298df02 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9d52 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.276576:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a64fa000/0xebdcd48a4298ebea lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc4c0 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276608:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x36:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276610:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x28:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276620:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7c00/0xebdcd48a4298ebc7 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc465 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276622:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890855000/0xebdcd48a4298dedf lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9cf7 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276639:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x35:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276653:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x27:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276654:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6800/0xebdcd48a4298eba4 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc40a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276662:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471200/0xebdcd48a4298debc lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9c9c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276668:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x34:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276674:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x26:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276679:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881639c00/0xebdcd48a4298eb81 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc3af expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276685:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471400/0xebdcd48a4298de99 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9c41 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276691:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x33:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276697:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x25:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276705:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7e00/0xebdcd48a4298eb5e lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc354 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276708:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470600/0xebdcd48a4298de76 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9be6 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276740:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x24:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.276742:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x32:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276745:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470200/0xebdcd48a4298de53 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9b8b expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276756:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x23:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276758:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7600/0xebdcd48a4298eb3b lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc2f9 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276763:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470c00/0xebdcd48a4298de30 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9b30 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276787:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x22:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.276788:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x31:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276792:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470e00/0xebdcd48a4298de0d lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9ad5 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276801:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x21:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276802:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6400/0xebdcd48a4298eb18 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc29e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276808:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967400/0xebdcd48a4298ddea lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9a7a expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276823:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x30:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276828:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x20:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.276848:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677b400/0xebdcd48a4298ddc7 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9a1f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.276858:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x1f:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276859:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6a00/0xebdcd48a4298eaf5 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc243 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276868:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677b800/0xebdcd48a4298dda4 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d99c4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276873:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x2f:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276884:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x1e:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276886:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6000/0xebdcd48a4298ead2 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc1e8 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276894:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0600/0xebdcd48a4298dd81 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9969 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276897:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x2e:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276912:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x1d:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276912:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36e00/0xebdcd48a4298eaaf lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc18d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276923:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1600/0xebdcd48a4298dd5e lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d990e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276929:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x2d:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276935:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x1c:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276940:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3400/0xebdcd48a4298ea8c lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc132 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276945:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1c00/0xebdcd48a4298dd3b lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d98b3 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276951:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x2c:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276958:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x1b:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276963:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2400/0xebdcd48a4298ea69 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc0d7 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276969:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0000/0xebdcd48a4298dd18 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9858 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276974:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x2b:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.276984:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x1a:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.276987:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2000/0xebdcd48a4298ea46 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc07c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.276994:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854000/0xebdcd48a4298dcf5 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d97fd expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.276998:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x2a:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277009:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3a00/0xebdcd48a4298ea23 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dc021 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277013:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x19:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277028:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854800/0xebdcd48a4298dcd2 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d97a2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277028:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x29:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277037:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978a00/0xebdcd48a4298ea00 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbfc6 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277041:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x18:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277055:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x28:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277056:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0e00/0xebdcd48a4298dcaf lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9747 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277065:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3800/0xebdcd48a4298e9dd lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbf6b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277068:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x17:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277081:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1400/0xebdcd48a4298dc8c lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d96ec expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277083:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x27:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277096:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718400/0xebdcd48a4298e9ba lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbf10 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277098:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x16:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277109:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1000/0xebdcd48a4298dc69 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9691 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277114:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x26:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.277120:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x15:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277124:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36000/0xebdcd48a4298e997 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbeb5 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277131:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0200/0xebdcd48a4298dc46 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9636 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277136:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x25:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.277147:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x14:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277149:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978200/0xebdcd48a4298e974 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbe5a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277158:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854600/0xebdcd48a4298dc23 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d95db expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277161:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x24:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277174:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978600/0xebdcd48a4298e951 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbdff expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277175:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x13:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277189:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677aa00/0xebdcd48a4298dc00 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9580 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277191:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x23:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277201:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890979e00/0xebdcd48a4298e92e lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbda4 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277212:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x12:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277215:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x22:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277224:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677a400/0xebdcd48a4298dbdd lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9525 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277226:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890979c00/0xebdcd48a4298e90b lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbd49 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277255:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x11:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277256:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x21:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277269:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677ae00/0xebdcd48a4298dbba lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d94ca expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277270:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978400/0xebdcd48a4298e8e8 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbcee expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277295:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x10:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277300:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x20:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277305:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677ba00/0xebdcd48a4298db97 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d946f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277314:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36600/0xebdcd48a4298e8c5 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbc93 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277317:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0xf:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277336:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890966c00/0xebdcd48a4298db74 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9414 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277357:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0xe:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277357:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x1f:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277371:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bce00/0xebdcd48a4298db51 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d93b9 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277374:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37600/0xebdcd48a4298e8a2 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbc38 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277379:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0xd:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277388:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x1e:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277390:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967e00/0xebdcd48a4298db2e lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d935e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277398:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36200/0xebdcd48a4298e87f lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbbdd expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277401:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0xc:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277417:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890966e00/0xebdcd48a4298db0b lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9303 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277417:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x1d:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277426:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36400/0xebdcd48a4298e85c lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbb82 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277434:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0xb:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277438:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x1c:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277446:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967800/0xebdcd48a4298dae8 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d92a8 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277449:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36800/0xebdcd48a4298e839 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbb27 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277463:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0xa:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277470:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x1b:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277474:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bdc00/0xebdcd48a4298dac5 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d924d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277484:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37200/0xebdcd48a4298e816 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dbacc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277486:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x9:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277498:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc200/0xebdcd48a4298daa2 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d91f2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277504:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x1a:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.277510:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x8:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277518:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37c00/0xebdcd48a4298e7f3 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dba71 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277521:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc800/0xebdcd48a4298da7f lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9197 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277542:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x19:0x0]: losing layout lock 00000080:00200000:0.0:1787827843.277543:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x7:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277558:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718e00/0xebdcd48a4298e7d0 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8dba16 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277559:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd400/0xebdcd48a4298da5c lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d913c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277583:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x6:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277593:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938e00/0xebdcd48a4298da39 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d90e1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277594:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x18:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277609:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550a00/0xebdcd48a4298e7ad lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db9bb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277610:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x5:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277624:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938600/0xebdcd48a4298da16 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d9086 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277625:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x17:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277636:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551800/0xebdcd48a4298e78a lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db960 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277643:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x4:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277649:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x16:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277654:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd000/0xebdcd48a4298d9ec lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8d900f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277660:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551600/0xebdcd48a4298e767 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db905 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787827843.277665:0:48516:0:(vvp_object.c:117:vvp_conf_set()) [0x200001b72:0x3:0x0]: losing layout lock 00000080:00200000:1.0:1787827843.277673:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x15:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277677:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc600/0xebdcd48a4298da01 lrc: 2/0,0 mode: PR/PR res: [0x200001b72:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8d9040 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.277683:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719e00/0xebdcd48a4298e744 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db8aa expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277693:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890939c00/0xebdcd48a4298d9ad lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8d8f28 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277695:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x14:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277707:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550600/0xebdcd48a4298e721 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db84f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277716:0:48516:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938000/0xebdcd48a4298d9a6 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8d8f0c expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827843.277719:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x13:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277731:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551200/0xebdcd48a4298e6fe lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db7f4 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277741:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x12:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277747:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551c00/0xebdcd48a4298e6db lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db799 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277757:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x11:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277762:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551400/0xebdcd48a4298e6b8 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db73e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277767:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x10:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277771:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719400/0xebdcd48a4298e695 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db6e3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277778:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0xf:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277779:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504400/0xebdcd48a4298e4ab lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8db1cd expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827843.277782:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550000/0xebdcd48a4298e672 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db688 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277802:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0xe:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277804:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890939200/0xebdcd48a4298d9d7 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d8fc2 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827843.277808:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a677b200/0xebdcd48a4298e64f lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db62d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277828:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827843.277850:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0xd:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277858:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890ba2800/0xebdcd48a4298e62c lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db5d2 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277864:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 1/0,0 mode: --/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787827843.277869:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0xc:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.277877:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887978e00/0xebdcd48a4298e609 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db577 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277878:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787827843.277898:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0xb:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277905:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277908:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719600/0xebdcd48a4298e5e6 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db51c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277919:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277938:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2e00/0xebdcd48a4298e411 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daa79 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277939:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0xa:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277947:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2200/0xebdcd48a4298e3ee lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daa1e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277948:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718200/0xebdcd48a4298e5c3 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db4c1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277965:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2c00/0xebdcd48a4298e3cb lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da9c3 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277968:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x9:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.277976:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2800/0xebdcd48a4298e3a8 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da968 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.277977:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a646e000/0xebdcd48a4298e5a0 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db466 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.277993:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1c00/0xebdcd48a4298e385 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da90d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.277995:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x8:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.278002:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887979200/0xebdcd48a4298e57d lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db40b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278003:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880030c00/0xebdcd48a4298e362 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da8b2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278020:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b13600/0xebdcd48a4298e33f lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da857 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.278020:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x7:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.278027:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881504800/0xebdcd48a4298e55a lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db3b0 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278029:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12600/0xebdcd48a4298e31c lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da7fc expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.278041:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x6:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.278043:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979600/0xebdcd48a4298e2f9 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da7a1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278048:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e537 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db355 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278050:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031a00/0xebdcd48a4298e2d6 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da746 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.278060:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x5:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.278062:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887978000/0xebdcd48a4298e2b3 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da6eb expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278067:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887978600/0xebdcd48a4298e514 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db2fa expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278069:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979000/0xebdcd48a4298e290 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x42:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da690 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278085:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148ec00/0xebdcd48a4298e26d lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da635 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.278086:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x4:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.278093:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7884b13a00/0xebdcd48a4298e4f1 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db29f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278095:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1200/0xebdcd48a4298e24a lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da5da expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278117:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0a00/0xebdcd48a4298e227 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da57f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.278120:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x3:0x0]: losing layout lock 00010000:00010000:0.0:1787827843.278127:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1600/0xebdcd48a4298e204 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da524 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278129:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881505a00/0xebdcd48a4298e4c0 lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x4059aa0aef8db21a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278140:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0800/0xebdcd48a4298e1e1 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da4c9 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787827843.278142:0:48517:0:(vvp_object.c:117:vvp_conf_set()) [0x240001b71:0x2:0x0]: losing layout lock 00010000:00010000:1.0:1787827843.278149:0:48517:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881505600/0xebdcd48a4298e4dc lrc: 2/0,0 mode: PR/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x4059aa0aef8db259 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827843.278151:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979a00/0xebdcd48a4298e1be lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da46e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278166:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887978c00/0xebdcd48a4298e19b lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da413 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278171:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979c00/0xebdcd48a4298e178 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x3a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da3b8 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278175:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979400/0xebdcd48a4298e155 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da35d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278179:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827843.278181:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031800/0xebdcd48a4298e132 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da302 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278200:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 1/0,0 mode: --/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827843.278201:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1e00/0xebdcd48a4298e10f lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da2a7 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278219:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827843.278220:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0400/0xebdcd48a4298e0ec lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da24c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278238:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278240:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0000/0xebdcd48a4298e0c9 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da1f1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278256:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278258:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148fa00/0xebdcd48a4298e0a6 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da196 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278276:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278277:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470800/0xebdcd48a4298e083 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da13b expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278297:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890855400/0xebdcd48a4298e060 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da0e0 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278300:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278305:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148f800/0xebdcd48a4298e03d lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da085 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278330:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148ee00/0xebdcd48a4298e01a lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da02a expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278341:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278343:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e600/0xebdcd48a4298dff7 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9fcf expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278384:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278388:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e400/0xebdcd48a4298dfd4 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9f74 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278421:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278424:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e000/0xebdcd48a4298dfb1 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9f19 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278453:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278456:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd200/0xebdcd48a4298df8e lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9ebe expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278492:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278494:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471a00/0xebdcd48a4298df6b lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9e63 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278536:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471e00/0xebdcd48a4298df48 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9e08 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278543:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18600/0xebdcd48a4298edf7 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dca15 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278549:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031000/0xebdcd48a4298df25 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9dad expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278595:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18a00/0xebdcd48a4298edd4 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc9ba expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278596:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031200/0xebdcd48a4298df02 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9d52 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278616:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18200/0xebdcd48a4298edb1 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc95f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278617:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890855000/0xebdcd48a4298dedf lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9cf7 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278635:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ed8e lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x42:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc904 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278636:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471200/0xebdcd48a4298debc lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9c9c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278655:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19c00/0xebdcd48a4298ed6b lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc8a9 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278656:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471400/0xebdcd48a4298de99 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9c41 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278675:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19200/0xebdcd48a4298ed48 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc84e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278676:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470600/0xebdcd48a4298de76 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9be6 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278695:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278696:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470200/0xebdcd48a4298de53 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9b8b expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278712:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278714:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470c00/0xebdcd48a4298de30 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9b30 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278728:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278729:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470e00/0xebdcd48a4298de0d lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9ad5 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278743:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18c00/0xebdcd48a4298ecbc lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc6e2 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278745:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967400/0xebdcd48a4298ddea lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9a7a expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278758:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2e00/0xebdcd48a4298ec99 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc687 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278760:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677b400/0xebdcd48a4298ddc7 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9a1f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278774:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2800/0xebdcd48a4298ec76 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x3a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc62c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278776:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677b800/0xebdcd48a4298dda4 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d99c4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278787:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278789:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0600/0xebdcd48a4298dd81 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9969 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278801:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21dbe00/0xebdcd48a4298ec30 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc576 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278803:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1600/0xebdcd48a4298dd5e lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d990e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278814:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da200/0xebdcd48a4298ec0d lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc51b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278817:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1c00/0xebdcd48a4298dd3b lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d98b3 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278829:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a64fa000/0xebdcd48a4298ebea lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc4c0 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278830:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0000/0xebdcd48a4298dd18 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9858 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278850:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854000/0xebdcd48a4298dcf5 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d97fd expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278854:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7c00/0xebdcd48a4298ebc7 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc465 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278857:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854800/0xebdcd48a4298dcd2 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d97a2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278880:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0e00/0xebdcd48a4298dcaf lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9747 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278899:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6800/0xebdcd48a4298eba4 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc40a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278900:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1400/0xebdcd48a4298dc8c lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d96ec expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278925:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1000/0xebdcd48a4298dc69 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9691 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278927:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881639c00/0xebdcd48a4298eb81 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc3af expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278933:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0200/0xebdcd48a4298dc46 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9636 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278945:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7e00/0xebdcd48a4298eb5e lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc354 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278947:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854600/0xebdcd48a4298dc23 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d95db expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278968:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677aa00/0xebdcd48a4298dc00 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9580 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278970:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7600/0xebdcd48a4298eb3b lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc2f9 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278981:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677a400/0xebdcd48a4298dbdd lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9525 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278983:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6400/0xebdcd48a4298eb18 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc29e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.278993:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677ae00/0xebdcd48a4298dbba lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d94ca expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.278996:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6a00/0xebdcd48a4298eaf5 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc243 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279006:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677ba00/0xebdcd48a4298db97 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d946f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279008:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6000/0xebdcd48a4298ead2 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc1e8 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279025:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890966c00/0xebdcd48a4298db74 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9414 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279026:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36e00/0xebdcd48a4298eaaf lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc18d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279046:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bce00/0xebdcd48a4298db51 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d93b9 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279047:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3400/0xebdcd48a4298ea8c lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc132 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279070:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967e00/0xebdcd48a4298db2e lrc: 1/0,0 mode: --/PR res: [0x200001b72:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d935e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279070:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2400/0xebdcd48a4298ea69 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc0d7 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279092:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2000/0xebdcd48a4298ea46 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc07c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279094:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890966e00/0xebdcd48a4298db0b lrc: 1/0,0 mode: --/PR res: [0x200001b72:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9303 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279105:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3a00/0xebdcd48a4298ea23 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc021 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279108:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967800/0xebdcd48a4298dae8 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d92a8 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279113:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978a00/0xebdcd48a4298ea00 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbfc6 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279118:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bdc00/0xebdcd48a4298dac5 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d924d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279121:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3800/0xebdcd48a4298e9dd lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbf6b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279126:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc200/0xebdcd48a4298daa2 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d91f2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279129:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718400/0xebdcd48a4298e9ba lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbf10 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279135:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc800/0xebdcd48a4298da7f lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9197 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279138:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36000/0xebdcd48a4298e997 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbeb5 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279144:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd400/0xebdcd48a4298da5c lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d913c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279148:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978200/0xebdcd48a4298e974 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbe5a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279154:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938e00/0xebdcd48a4298da39 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d90e1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279157:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978600/0xebdcd48a4298e951 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbdff expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279163:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938600/0xebdcd48a4298da16 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9086 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279168:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890979e00/0xebdcd48a4298e92e lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbda4 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279172:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd000/0xebdcd48a4298d9ec lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d900f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279178:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890979c00/0xebdcd48a4298e90b lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbd49 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279182:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc600/0xebdcd48a4298da01 lrc: 1/0,0 mode: --/PR res: [0x200001b72:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d9040 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.279187:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978400/0xebdcd48a4298e8e8 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbcee expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279193:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890939c00/0xebdcd48a4298d9ad lrc: 1/0,0 mode: --/PR res: [0x200000002:0x3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d8f28 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279196:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36600/0xebdcd48a4298e8c5 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbc93 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279203:0:48516:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938000/0xebdcd48a4298d9a6 lrc: 1/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d8f0c expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.279206:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37600/0xebdcd48a4298e8a2 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbc38 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279244:0:48516:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 85 locks packed 00010000:00010000:1.0:1787827843.279249:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36200/0xebdcd48a4298e87f lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbbdd expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279263:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36400/0xebdcd48a4298e85c lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbb82 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279275:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36800/0xebdcd48a4298e839 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbb27 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279290:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37200/0xebdcd48a4298e816 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbacc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279297:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7881504400/0xebdcd48a4298e4ab lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8db1cd expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827843.279312:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37c00/0xebdcd48a4298e7f3 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dba71 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279364:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718e00/0xebdcd48a4298e7d0 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dba16 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279366:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890939200/0xebdcd48a4298d9d7 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d8fc2 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827843.279401:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550a00/0xebdcd48a4298e7ad lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db9bb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279409:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3e00/0xebdcd48a4298e481 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dabc2 expref: -99 pid: 48374 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.279412:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551800/0xebdcd48a4298e78a lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db960 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000100:00080000:3.0:1787827843.279428:0:10936:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff9a7890a3e580 x1874671476754176/t0(0) o103->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:17/18 lens 992/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787827843.279447:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551600/0xebdcd48a4298e767 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db905 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279451:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2000/0xebdcd48a4298e47a lrc: 0/0,0 mode: --/CR res: [0x200001b72:0x51:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daba6 expref: -99 pid: 48373 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827843.279477:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719e00/0xebdcd48a4298e744 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db8aa expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279484:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3c00/0xebdcd48a4298e46c lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x50:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8dab75 expref: -99 pid: 48373 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.279497:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550600/0xebdcd48a4298e721 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db84f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279502:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c3a00/0xebdcd48a4298e457 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x4f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dab2f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279526:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551200/0xebdcd48a4298e6fe lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db7f4 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279530:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2600/0xebdcd48a4298e434 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daad4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279558:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551c00/0xebdcd48a4298e6db lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db799 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279578:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2e00/0xebdcd48a4298e411 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daa79 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279588:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551400/0xebdcd48a4298e6b8 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db73e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279593:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2200/0xebdcd48a4298e3ee lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8daa1e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279595:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719400/0xebdcd48a4298e695 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db6e3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279613:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550000/0xebdcd48a4298e672 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db688 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279627:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a677b200/0xebdcd48a4298e64f lrc: 1/0,0 mode: --/PR res: [0x240001b71:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db62d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279632:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890ba2800/0xebdcd48a4298e62c lrc: 1/0,0 mode: --/PR res: [0x240001b71:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db5d2 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279635:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2c00/0xebdcd48a4298e3cb lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da9c3 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279637:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887978e00/0xebdcd48a4298e609 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db577 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279645:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719600/0xebdcd48a4298e5e6 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db51c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279658:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718200/0xebdcd48a4298e5c3 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db4c1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279660:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78908c2800/0xebdcd48a4298e3a8 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da968 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279665:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a646e000/0xebdcd48a4298e5a0 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db466 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279673:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1c00/0xebdcd48a4298e385 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da90d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279674:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887979200/0xebdcd48a4298e57d lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db40b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279694:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881504800/0xebdcd48a4298e55a lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db3b0 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279697:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880030c00/0xebdcd48a4298e362 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da8b2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279701:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e537 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db355 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279711:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887978600/0xebdcd48a4298e514 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db2fa expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279712:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b13600/0xebdcd48a4298e33f lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da857 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279726:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7884b13a00/0xebdcd48a4298e4f1 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db29f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279732:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881505a00/0xebdcd48a4298e4c0 lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db21a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279734:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7884b12600/0xebdcd48a4298e31c lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da7fc expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279741:0:48517:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881505600/0xebdcd48a4298e4dc lrc: 1/0,0 mode: --/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8db259 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.279749:0:48517:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 81 locks packed 00010000:00010000:0.0:1787827843.279750:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979600/0xebdcd48a4298e2f9 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da7a1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279761:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031a00/0xebdcd48a4298e2d6 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da746 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279768:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace400/0xebdcd48a4298ef5c lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x51:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcddb expref: -99 pid: 48388 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827843.279770:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887978000/0xebdcd48a4298e2b3 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da6eb expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000100:00080000:2.0:1787827843.279794:0:10937:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff9a7890b1cf00 x1874671476754304/t0(0) o103->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:17/18 lens 960/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787827843.279802:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace200/0xebdcd48a4298ef55 lrc: 0/0,0 mode: --/CR res: [0x240001b71:0x50:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcdbf expref: -99 pid: 48387 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787827843.279803:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979000/0xebdcd48a4298e290 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x42:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da690 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279860:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf600/0xebdcd48a4298ef47 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8dcd8e expref: -99 pid: 48387 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787827843.279865:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148ec00/0xebdcd48a4298e26d lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da635 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279878:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05c00/0xebdcd48a4298ef32 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x4e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcd48 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279882:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1200/0xebdcd48a4298e24a lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da5da expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279897:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886ace000/0xebdcd48a4298ef0f lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x4d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcced expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279906:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0a00/0xebdcd48a4298e227 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da57f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279915:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886acf000/0xebdcd48a4298eeec lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x4c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcc92 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279923:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1600/0xebdcd48a4298e204 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da524 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279929:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04800/0xebdcd48a4298eec9 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x4b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcc37 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279937:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0800/0xebdcd48a4298e1e1 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da4c9 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279944:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05000/0xebdcd48a4298eea6 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x4a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcbdc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279954:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979a00/0xebdcd48a4298e1be lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da46e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279958:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a05400/0xebdcd48a4298ee83 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x49:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcb81 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.279974:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887978c00/0xebdcd48a4298e19b lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da413 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.279977:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7885a04000/0xebdcd48a4298ee60 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x48:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcb26 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280001:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979c00/0xebdcd48a4298e178 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x3a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da3b8 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280003:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18400/0xebdcd48a4298ee3d lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x47:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dcacb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280025:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18e00/0xebdcd48a4298ee1a lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x46:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dca70 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280032:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7887979400/0xebdcd48a4298e155 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da35d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280039:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18600/0xebdcd48a4298edf7 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x45:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dca15 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280046:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031800/0xebdcd48a4298e132 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da302 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280053:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18a00/0xebdcd48a4298edd4 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc9ba expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280060:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e1e00/0xebdcd48a4298e10f lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da2a7 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280082:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18200/0xebdcd48a4298edb1 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc95f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280084:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0400/0xebdcd48a4298e0ec lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da24c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280115:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18800/0xebdcd48a4298ed8e lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x42:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc904 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280116:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78869e0000/0xebdcd48a4298e0c9 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da1f1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280148:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19c00/0xebdcd48a4298ed6b lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x41:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc8a9 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280149:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148fa00/0xebdcd48a4298e0a6 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da196 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280179:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a19200/0xebdcd48a4298ed48 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc84e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280180:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470800/0xebdcd48a4298e083 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da13b expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280213:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907600/0xebdcd48a4298ed25 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc7f3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280214:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890855400/0xebdcd48a4298e060 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da0e0 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280269:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890906600/0xebdcd48a4298ed02 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x3e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc798 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280272:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148f800/0xebdcd48a4298e03d lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da085 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280372:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907400/0xebdcd48a4298ecdf lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x3d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc73d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280373:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148ee00/0xebdcd48a4298e01a lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8da02a expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280397:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e600/0xebdcd48a4298dff7 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9fcf expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280402:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890a18c00/0xebdcd48a4298ecbc lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc6e2 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280427:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e400/0xebdcd48a4298dfd4 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9f74 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280438:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2e00/0xebdcd48a4298ec99 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x3b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc687 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280447:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a788148e000/0xebdcd48a4298dfb1 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9f19 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280496:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd200/0xebdcd48a4298df8e lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9ebe expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280500:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2800/0xebdcd48a4298ec76 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x3a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc62c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280511:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471a00/0xebdcd48a4298df6b lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9e63 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280537:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890907a00/0xebdcd48a4298ec53 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc5d1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280538:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471e00/0xebdcd48a4298df48 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9e08 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280590:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031000/0xebdcd48a4298df25 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9dad expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280596:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21dbe00/0xebdcd48a4298ec30 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc576 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280601:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7880031200/0xebdcd48a4298df02 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9d52 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280618:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78c21da200/0xebdcd48a4298ec0d lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc51b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280623:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890855000/0xebdcd48a4298dedf lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9cf7 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280636:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a64fa000/0xebdcd48a4298ebea lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc4c0 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280639:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471200/0xebdcd48a4298debc lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9c9c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280660:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7c00/0xebdcd48a4298ebc7 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc465 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280662:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888471400/0xebdcd48a4298de99 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9c41 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280692:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6800/0xebdcd48a4298eba4 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc40a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280693:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470600/0xebdcd48a4298de76 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9be6 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280722:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881639c00/0xebdcd48a4298eb81 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc3af expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280724:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470200/0xebdcd48a4298de53 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9b8b expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280756:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7e00/0xebdcd48a4298eb5e lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc354 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280758:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470c00/0xebdcd48a4298de30 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9b30 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280782:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7888470e00/0xebdcd48a4298de0d lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9ad5 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280788:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c7600/0xebdcd48a4298eb3b lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc2f9 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280794:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967400/0xebdcd48a4298ddea lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9a7a expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280801:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6400/0xebdcd48a4298eb18 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc29e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280809:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677b400/0xebdcd48a4298ddc7 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9a1f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280815:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6a00/0xebdcd48a4298eaf5 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc243 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280847:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677b800/0xebdcd48a4298dda4 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d99c4 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280848:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a65c6000/0xebdcd48a4298ead2 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc1e8 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280896:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0600/0xebdcd48a4298dd81 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9969 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280902:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36e00/0xebdcd48a4298eaaf lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc18d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280912:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1600/0xebdcd48a4298dd5e lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d990e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280917:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3400/0xebdcd48a4298ea8c lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc132 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280925:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1c00/0xebdcd48a4298dd3b lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d98b3 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280931:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2400/0xebdcd48a4298ea69 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc0d7 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280941:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0000/0xebdcd48a4298dd18 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9858 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280946:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd2000/0xebdcd48a4298ea46 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc07c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280957:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854000/0xebdcd48a4298dcf5 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d97fd expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280962:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3a00/0xebdcd48a4298ea23 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dc021 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280973:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854800/0xebdcd48a4298dcd2 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d97a2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280981:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978a00/0xebdcd48a4298ea00 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbfc6 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.280987:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0e00/0xebdcd48a4298dcaf lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9747 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.280994:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886bd3800/0xebdcd48a4298e9dd lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbf6b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281001:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1400/0xebdcd48a4298dc8c lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d96ec expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281008:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718400/0xebdcd48a4298e9ba lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbf10 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281032:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e1000/0xebdcd48a4298dc69 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9691 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281033:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36000/0xebdcd48a4298e997 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbeb5 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281070:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a64e0200/0xebdcd48a4298dc46 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9636 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281072:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978200/0xebdcd48a4298e974 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbe5a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281108:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890854600/0xebdcd48a4298dc23 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d95db expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281108:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978600/0xebdcd48a4298e951 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbdff expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281148:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677aa00/0xebdcd48a4298dc00 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9580 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281148:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890979e00/0xebdcd48a4298e92e lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbda4 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281177:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890979c00/0xebdcd48a4298e90b lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbd49 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281180:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677a400/0xebdcd48a4298dbdd lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9525 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281200:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890978400/0xebdcd48a4298e8e8 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbcee expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281202:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677ae00/0xebdcd48a4298dbba lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d94ca expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281228:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78a677ba00/0xebdcd48a4298db97 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d946f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281234:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36600/0xebdcd48a4298e8c5 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbc93 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281241:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890966c00/0xebdcd48a4298db74 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9414 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281248:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37600/0xebdcd48a4298e8a2 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbc38 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281255:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bce00/0xebdcd48a4298db51 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d93b9 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281262:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36200/0xebdcd48a4298e87f lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbbdd expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281282:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967e00/0xebdcd48a4298db2e lrc: 0/0,0 mode: --/PR res: [0x200001b72:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d935e expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281288:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36400/0xebdcd48a4298e85c lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbb82 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281296:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890966e00/0xebdcd48a4298db0b lrc: 0/0,0 mode: --/PR res: [0x200001b72:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9303 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281309:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b36800/0xebdcd48a4298e839 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbb27 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281312:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890967800/0xebdcd48a4298dae8 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d92a8 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281354:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37200/0xebdcd48a4298e816 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dbacc expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281355:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bdc00/0xebdcd48a4298dac5 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d924d expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281390:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7886b37c00/0xebdcd48a4298e7f3 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dba71 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281391:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc200/0xebdcd48a4298daa2 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d91f2 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281415:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc800/0xebdcd48a4298da7f lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9197 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281423:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718e00/0xebdcd48a4298e7d0 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8dba16 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281428:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd400/0xebdcd48a4298da5c lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d913c expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281458:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938e00/0xebdcd48a4298da39 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d90e1 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281458:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550a00/0xebdcd48a4298e7ad lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db9bb expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281485:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938600/0xebdcd48a4298da16 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d9086 expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281502:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551800/0xebdcd48a4298e78a lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db960 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281503:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bd000/0xebdcd48a4298d9ec lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8d900f expref: -99 pid: 48370 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281530:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a78909bc600/0xebdcd48a4298da01 lrc: 0/0,0 mode: --/PR res: [0x200001b72:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d9040 expref: -99 pid: 48370 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.281536:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551600/0xebdcd48a4298e767 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db905 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281543:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890939c00/0xebdcd48a4298d9ad lrc: 0/0,0 mode: --/PR res: [0x200000002:0x3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d8f28 expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281550:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719e00/0xebdcd48a4298e744 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db8aa expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281556:0:48516:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9a78a6512000 lock: ffff9a7890938000/0xebdcd48a4298d9a6 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8d8f0c expref: -99 pid: 48199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.281565:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550600/0xebdcd48a4298e721 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db84f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787827843.281570:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827843.281574:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827843.281577:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827843.281581:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551200/0xebdcd48a4298e6fe lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db7f4 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281590:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551c00/0xebdcd48a4298e6db lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db799 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281597:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883551400/0xebdcd48a4298e6b8 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db73e expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281603:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719400/0xebdcd48a4298e695 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db6e3 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281609:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7883550000/0xebdcd48a4298e672 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db688 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281619:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a677b200/0xebdcd48a4298e64f lrc: 0/0,0 mode: --/PR res: [0x240001b71:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db62d expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281626:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7890ba2800/0xebdcd48a4298e62c lrc: 0/0,0 mode: --/PR res: [0x240001b71:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db5d2 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281632:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887978e00/0xebdcd48a4298e609 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db577 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281641:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888719600/0xebdcd48a4298e5e6 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db51c expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281651:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7888718200/0xebdcd48a4298e5c3 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db4c1 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281662:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a78a646e000/0xebdcd48a4298e5a0 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db466 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281673:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887979200/0xebdcd48a4298e57d lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db40b expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281686:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881504800/0xebdcd48a4298e55a lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db3b0 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281698:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7884b12400/0xebdcd48a4298e537 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db355 expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281715:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7887978600/0xebdcd48a4298e514 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db2fa expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281726:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7884b13a00/0xebdcd48a4298e4f1 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db29f expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281734:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881505a00/0xebdcd48a4298e4c0 lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x4059aa0aef8db21a expref: -99 pid: 48385 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787827843.281741:0:48517:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9a78a6512000 lock: ffff9a7881505600/0xebdcd48a4298e4dc lrc: 0/0,0 mode: --/PR res: [0x240001b71:0x1:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x4059aa0aef8db259 expref: -99 pid: 48385 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787827843.281748:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827843.281749:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827843.281750:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827844.297947:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827845.321920:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827845.321928:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827845.321932:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827845.321940:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2212 00000100:00080000:1.0:1787827845.321949:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827845.321953:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827845.321956:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827845.321986:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476754432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827845.322007:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827845.322010:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827845.322012:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827845.322014:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2212 00000100:00080000:1.0:1787827845.322017:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827845.322020:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827845.322028:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827845.322038:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476754560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827845.322046:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476754432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827845.322059:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827845.322062:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827845.322063:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827845.322067:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2212 00000100:00080000:1.0:1787827845.322069:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827845.322072:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827845.322075:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827845.322084:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476754688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827845.322091:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827845.322094:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827845.322095:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827845.322098:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2212 00000100:00080000:1.0:1787827845.322100:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827845.322102:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827845.322105:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827845.322112:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827845.322114:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476754816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827845.322122:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827845.322124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827845.322125:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827845.322128:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2212 00000100:00080000:1.0:1787827845.322133:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827845.322135:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827845.322138:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827845.322145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476754944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827845.322166:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9a7890b1d680 x1874671476754432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827845.322183:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476754432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827845.322194:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827845.322198:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827845.322202:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476754432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827845.322222:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827845.322242:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827845.322254:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9a7890b1c3c0 x1874671476754560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827845.322265:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476754560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827845.322273:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827845.322276:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827845.322279:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827845.322285:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827845.322301:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827845.322325:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9a7890b1de00 x1874671476754688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827845.322336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476754688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827845.322343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827845.322346:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827845.322349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827845.322355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827845.322372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827845.322381:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9a7890b1e580 x1874671476754816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827845.322391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476754816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827845.322400:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827845.322403:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827845.322406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827845.322411:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827845.322425:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827845.322434:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9a7890b1da40 x1874671476754944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827845.322445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476754944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827845.322455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827845.322458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827845.322461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476754944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827846.346023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827847.369940:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827848.393968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827849.417991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827850.442009:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827850.442022:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:1.0:1787827850.442025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827850.442029:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827850.442067:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2217 00000100:00080000:3.0:1787827850.442080:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827850.442087:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827850.442159:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476755072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827850.442190:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827850.442195:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827850.442197:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827850.442201:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2217 00000100:00080000:3.0:1787827850.442205:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827850.442209:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827850.442227:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827850.442243:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476755200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827850.442256:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827850.442260:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827850.442265:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827850.442268:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476755072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827850.442290:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2217 00000100:00080000:3.0:1787827850.442300:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827850.442305:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827850.442309:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827850.442326:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476755328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827850.442336:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827850.442341:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827850.442344:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827850.442348:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2217 00000100:00080000:3.0:1787827850.442371:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827850.442375:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827850.442380:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827850.442387:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827850.442398:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476755456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827850.442412:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827850.442416:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827850.442418:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827850.442422:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2217 00000100:00080000:3.0:1787827850.442427:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827850.442432:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827850.442438:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827850.442456:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476755584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827850.442464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9a78a6776940 x1874671476755072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827850.442492:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476755072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827850.442519:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827850.442524:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827850.442528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476755072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827850.442555:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827850.442597:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827850.442611:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9a78a6776580 x1874671476755200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827850.442625:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476755200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827850.442634:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827850.442638:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827850.442641:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827850.442647:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827850.442665:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827850.442679:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9a78a6774000 x1874671476755328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827850.442698:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476755328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827850.442711:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827850.442717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827850.442739:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827850.442747:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827850.442771:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827850.442782:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9a78a67752c0 x1874671476755456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827850.442791:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476755456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827850.442800:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827850.442803:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827850.442805:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827850.442810:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827850.442843:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827850.442853:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827850/real 1787827850] req@ffff9a78a6775a40 x1874671476755584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827850.442865:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476755584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827850.442875:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827850.442927:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827850.442931:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827851.465944:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827852.490208:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827853.514029:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827854.537938:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827855.561907:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827855.561915:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827855.561918:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827855.561926:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2223 00000100:00080000:3.0:1787827855.561935:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827855.561940:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827855.561972:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476755712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827855.561993:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827855.561996:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827855.561998:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827855.562000:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2223 00000100:00080000:3.0:1787827855.562003:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827855.562005:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827855.562013:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827855.562022:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476755840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827855.562033:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476755712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827855.562035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827855.562045:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827855.562048:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827855.562053:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2223 00000100:00080000:3.0:1787827855.562057:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827855.562062:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827855.562068:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827855.562083:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476755968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827855.562094:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827855.562096:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827855.562098:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827855.562101:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2223 00000100:00080000:3.0:1787827855.562105:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827855.562107:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827855.562112:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827855.562124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476756096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827855.562130:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827855.562132:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827855.562134:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827855.562139:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755840:192.168.203.140@tcp:38:kworker.0 00000100:00080000:3.0:1787827855.562139:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2223 00000100:00080000:3.0:1787827855.562145:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827855.562149:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827855.562153:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827855.562164:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476756224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827855.562172:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827855.562193:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827855.562216:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827855.562296:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827856.586025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827857.609951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827858.633995:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827858.634342:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787827858.634407:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827858.634432:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827858.634498:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827858.634511:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827858.634522:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787827858.634533:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827858.634627:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827855/real 1787827858] req@ffff9a78a6774000 x1874671476756224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827858.634679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476756224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827858.634706:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827858.634717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827858.634726:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756224:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827858.634741:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827855/real 1787827858] req@ffff9a78a6777480 x1874671476755840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827858.634767:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476755840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827858.634784:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827858.634791:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827858.634798:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827858.634812:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827855/real 1787827858] req@ffff9a78a6775a40 x1874671476755968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827858.634921:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476755968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827858.634939:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827858.634946:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827858.634953:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476755968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827858.634965:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827855/real 1787827858] req@ffff9a78a67752c0 x1874671476756096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827858.634988:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476756096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827858.635005:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827858.635011:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827858.635017:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827858.635028:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827855/real 1787827858] req@ffff9a78a6774b40 x1874671476755712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827858.635050:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476755712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827858.635063:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827858.635069:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827858.635074:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476755712:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787827859.657979:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00020000:2.0:1787827860.681915:0:38:0:(osc_cache.c:999:osc_extent_wait()) extent ffff9a7884d9b600@{[0 -> 0/1023], [2|0|+|cache|wiuY|ffff9a78868fea80], [28672|1|+|-|ffff9a78a64e0c00|1024|0000000000000000]} lustre-OST0000-osc-ffff9a78a6512000: wait ext to 0 timedout, recovery in progress? 00000100:00080000:3.0:1787827860.681931:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827860.681941:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827860.681946:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827860.681957:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2228 00000100:00080000:3.0:1787827860.681969:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827860.681977:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827860.682012:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476756352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827860.682065:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827860.682071:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827860.682077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827860.682084:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2228 00000100:00080000:3.0:1787827860.682090:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827860.682097:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827860.682111:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476756352:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787827860.682111:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827860.682141:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476756480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827860.682173:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827860.682177:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827860.682179:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827860.682185:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2228 00000100:00080000:3.0:1787827860.682191:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787827860.682215:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827860.682216:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827860.682225:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827860.682246:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476756608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827860.682259:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827860.682263:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827860.682265:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827860.682271:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2228 00000100:00100000:1.0:1787827860.682276:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827860/real 1787827860] req@ffff9a78a6776580 x1874671476756352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827860.682278:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827860.682288:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827860.682301:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827860.682311:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476756352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827860.682318:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476756736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827860.682335:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827860.682338:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827860.682343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787827860.682345:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827860.682351:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827860.682353:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476756352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827860.682359:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2228 00000100:00080000:3.0:1787827860.682365:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827860.682370:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827860.682375:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827860.682387:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827860.682390:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476756864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827860.682421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827860.682441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827860/real 1787827860] req@ffff9a78a6776940 x1874671476756480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827860.682455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476756480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827860.682464:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827860.682468:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827860.682472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827860.682479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827860.682501:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827860.682513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827860/real 1787827860] req@ffff9a78a67770c0 x1874671476756608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827860.682532:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476756608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827860.682540:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827860.682544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827860.682547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827860.682554:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827860.682576:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827860.682589:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827860/real 1787827860] req@ffff9a78a6774780 x1874671476756736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827860.682602:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476756736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827860.682613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827860.682617:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827860.682621:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827860.682641:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827860.682661:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827860.682673:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827860/real 1787827860] req@ffff9a78a67761c0 x1874671476756864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827860.682686:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476756864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827860.682704:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827860.682708:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827860.682712:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476756864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827860.682780:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00020000:2.0:1787827860.695851:0:38:0:(osc_cache.c:999:osc_extent_wait()) ### extent: 000000002cff6c00 ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 00000000b63ab6bc/0xebdcd48a4298dc77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x6b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400020000 nid: local remote: 0x4059aa0aef8d96a6 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787827861.705989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827862.730126:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827863.754050:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827864.778142:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827865.801969:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827865.801980:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827865.801986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827865.801998:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2233 00000100:00080000:0.0:1787827865.802010:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827865.802017:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827865.802043:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827865.802061:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476756992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827865.802085:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827865.802088:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827865.802090:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827865.802093:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2233 00000100:00080000:0.0:1787827865.802097:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827865.802100:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827865.802110:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827865.802117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476756992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827865.802123:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476757120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827865.802132:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827865.802134:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827865.802136:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827865.802139:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2233 00000100:00080000:0.0:1787827865.802141:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827865.802143:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827865.802146:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827865.802155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476757248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827865.802182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827865.802185:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827865.802187:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827865.802190:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2233 00000100:00080000:0.0:1787827865.802199:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827865.802203:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827865.802208:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827865.802212:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827865.802222:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476757376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827865.802230:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827865.802232:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827865.802234:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827865.802238:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2233 00000100:00080000:0.0:1787827865.802240:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827865.802242:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827865.802245:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827865.802254:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476757504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827865.802281:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827865/real 1787827865] req@ffff9a7890a3c3c0 x1874671476756992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827865.802300:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476756992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827865.802309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827865.802313:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827865.802317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476756992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827865.802346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827865.802370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827865.802382:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827865/real 1787827865] req@ffff9a7890a3c000 x1874671476757120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827865.802390:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476757120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827865.802396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827865.802399:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827865.802402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827865.802406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827865.802419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827865.802425:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827865/real 1787827865] req@ffff9a7890a3f480 x1874671476757248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827865.802433:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476757248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827865.802438:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827865.802440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827865.802441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827865.802445:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827865.802455:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827865.802460:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827865/real 1787827865] req@ffff9a7890a3de00 x1874671476757376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827865.802468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476757376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827865.802475:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827865.802476:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827865.802478:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827865.802482:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827865.802491:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827865.802497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827865/real 1787827865] req@ffff9a7890a3da40 x1874671476757504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827865.802504:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476757504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827865.802511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827865.802512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827865.802514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827866.825978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827867.850093:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827868.873954:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827869.898064:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827870.921987:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827870.922016:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827870.922027:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827870.922032:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827870.922043:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2238 00000100:00080000:2.0:1787827870.922055:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827870.922062:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787827870.922103:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476757632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827870.922157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827870.922161:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827870.922163:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827870.922167:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2238 00000100:00080000:2.0:1787827870.922171:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827870.922176:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827870.922188:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827870.922203:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476757632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787827870.922204:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476757760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827870.922226:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827870.922229:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827870.922231:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827870.922235:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2238 00000100:00080000:2.0:1787827870.922239:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827870.922242:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827870.922246:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827870.922258:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476757888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827870.922268:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827870.922271:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827870.922273:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827870.922277:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2238 00000100:00080000:2.0:1787827870.922281:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827870.922284:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827870.922289:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827870.922291:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827870.922306:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476758016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827870.922317:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827870.922320:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827870.922321:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827870.922325:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2238 00000100:00080000:2.0:1787827870.922328:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827870.922332:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827870.922337:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827870.922350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827870/real 1787827870] req@ffff9a78841d8780 x1874671476757632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827870.922351:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476758144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827870.922408:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476757632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827870.922420:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827870.922424:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827870.922428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476757632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827870.922453:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827870.922474:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827870.922490:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827870/real 1787827870] req@ffff9a78841d9a40 x1874671476757760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827870.922510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476757760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827870.922522:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827870.922527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827870.922532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827870.922542:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827870.922569:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827870.922582:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827870/real 1787827870] req@ffff9a78841db0c0 x1874671476757888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827870.922593:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476757888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827870.922601:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827870.922604:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827870.922607:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476757888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827870.922613:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827870.922629:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827870.922637:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827870/real 1787827870] req@ffff9a78841da1c0 x1874671476758016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827870.922660:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476758016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827870.922679:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827870.922682:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827870.922685:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827870.922690:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827870.922706:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827870.922714:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827870/real 1787827870] req@ffff9a78841dbc00 x1874671476758144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827870.922724:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476758144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827895 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827870.922733:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827870.922735:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827870.922738:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827871.945997:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827872.970063:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827873.994040:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827875.017977:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827876.042119:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787827876.042140:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827876.042161:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827876.042172:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827876.042192:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2243 00000100:00080000:0.0:1787827876.042224:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827876.042239:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827876.042312:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476758272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827876.042386:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827876.042395:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827876.042400:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827876.042415:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2243 00000100:00080000:0.0:1787827876.042450:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827876.042460:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827876.042473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476758272:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787827876.042481:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827876.042516:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476758400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827876.042542:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827876.042550:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827876.042555:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827876.042564:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2243 00000100:00080000:0.0:1787827876.042572:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787827876.042581:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827876.042584:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827876.042599:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827876.042629:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476758528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827876.042658:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9a7890a3fc00 x1874671476758272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827876.042669:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827876.042687:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1787827876.042697:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476758272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827876.042698:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827876.042723:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2243 00000100:00080000:0.0:1787827876.042741:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827876.042752:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827876.042758:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827876.042764:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476758272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827876.042788:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827876.042800:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758400:192.168.203.140@tcp:38:kworker.0 00000008:00100000:0.0:1787827876.042802:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827876.042884:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827876.042902:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9a7890a3da40 x1874671476758400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827876.042913:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476758656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827876.042932:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476758400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827876.042996:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827876.042997:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827876.043011:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787827876.043021:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:1.0:1787827876.043024:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758400:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827876.043029:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827876.043041:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758528:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827876.043052:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2243 00000800:00000100:1.0:1787827876.043072:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827876.043081:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827876.043089:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9a7890a3de00 x1874671476758528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827876.043123:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827876.043146:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827876.043149:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476758528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827876.043167:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827876.043171:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827876.043177:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827876.043186:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476758784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827876.043206:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827876.043243:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827876.043257:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9a7890a3f480 x1874671476758656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827876.043270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476758656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827876.043281:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827876.043285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827876.043288:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827876.043300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827876.043316:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827876.043326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9a7890a3c000 x1874671476758784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827876.043337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476758784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827876.043346:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827876.043349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827876.043352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476758784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827877.065972:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827878.090021:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827879.114130:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827880.138001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827881.161955:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827881.161962:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827881.161967:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827881.161975:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2248 00010000:00010000:1.0:1787827881.161978:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827881.161985:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827881.161993:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827881.162027:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476758912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827881.162045:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827881.162048:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827881.162049:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827881.162052:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2248 00000100:00080000:3.0:1787827881.162054:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827881.162069:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827881.162077:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827881.162087:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476759040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827881.162097:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476758912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827881.162097:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827881.162104:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827881.162107:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827881.162111:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2248 00000100:00080000:3.0:1787827881.162114:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827881.162116:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827881.162120:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827881.162130:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476759168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827881.162149:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827881.162152:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827881.162154:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827881.162159:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2248 00000100:00080000:3.0:1787827881.162171:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827881.162175:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827881.162181:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787827881.162182:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827881.162197:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476759296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827881.162205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827881.162207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827881.162208:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827881.162211:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2248 00000100:00080000:3.0:1787827881.162214:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827881.162217:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827881.162220:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827881.162228:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476759424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827881.162258:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9a78a6776d00 x1874671476758912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827881.162279:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476758912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827881.162290:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827881.162295:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827881.162299:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476758912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827881.162324:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827881.162350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827881.162365:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9a78a6775680 x1874671476759040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827881.162378:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476759040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827881.162388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827881.162392:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827881.162395:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827881.162403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827881.162427:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827881.162441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9a78a6777840 x1874671476759168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827881.162454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476759168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827881.162463:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827881.162467:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827881.162470:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827881.162477:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827881.162497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827881.162509:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9a78a6774f00 x1874671476759296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827881.162522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476759296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827881.162532:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827881.162536:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827881.162539:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827881.162545:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827881.162565:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827881.162576:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827881/real 1787827881] req@ffff9a78a67743c0 x1874671476759424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827881.162595:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476759424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827881.162605:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827881.162609:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827881.162612:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827882.185999:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827883.209984:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827884.233971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827885.257961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827886.281876:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827886.281884:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827886.281888:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.281896:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2253 00000100:00080000:3.0:1787827886.281905:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.281910:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827886.281942:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476759552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827886.281969:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827886.281973:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827886.281975:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.281980:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2253 00000100:00080000:3.0:1787827886.281984:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.281990:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827886.282000:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827886.282007:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476759552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827886.282014:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476759680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827886.282034:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827886.282037:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827886.282039:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.282042:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2253 00000100:00080000:3.0:1787827886.282046:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.282050:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827886.282054:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827886.282065:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476759808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827886.282078:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827886.282084:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827886.282087:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827886.282089:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.282093:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2253 00000100:00080000:3.0:1787827886.282098:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.282100:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827886.282103:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827886.282114:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476759936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827886.282121:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827886.282125:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9a78a6777c00 x1874671476759552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827886.282125:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827886.282129:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.282136:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2253 00000100:00080000:3.0:1787827886.282141:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.282146:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827886.282149:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827886.282152:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476759552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827886.282160:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476760064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827886.282176:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.282184:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827886.282189:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476759552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827886.282216:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827886.282238:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827886.282249:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9a78a6775e00 x1874671476759680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827886.282265:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476759680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787827886.282267:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827886.282274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.282278:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827886.282281:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827886.282286:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827886.282297:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827886.282303:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9a78a67743c0 x1874671476759808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827886.282311:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476759808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827886.282316:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.282318:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827886.282320:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827886.282324:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827886.282337:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827886.282348:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9a78a6774f00 x1874671476759936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827886.282360:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476759936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827886.282377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.282380:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827886.282384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476759936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827886.282390:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827886.282406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827886.282414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9a78a6777840 x1874671476760064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827886.282422:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476760064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827886.282429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.282431:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827886.282433:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827887.305953:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827888.329935:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827889.353920:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827890.377937:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827891.402004:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.402030:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827891.402035:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.402048:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2258 00000100:00080000:3.0:1787827891.402063:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.402070:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827891.402111:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476760192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.402139:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.402143:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827891.402144:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.402147:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2258 00000100:00080000:3.0:1787827891.402149:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.402152:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827891.402161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827891.402171:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476760320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.402178:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.402181:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827891.402182:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.402185:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2258 00000100:00080000:3.0:1787827891.402189:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.402193:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827891.402198:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827891.402203:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476760192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827891.402216:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476760448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.402230:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.402235:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827891.402237:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.402242:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2258 00000100:00080000:3.0:1787827891.402246:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.402250:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827891.402255:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827891.402265:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476760576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.402272:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.402275:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827891.402276:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.402278:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2258 00000100:00080000:3.0:1787827891.402280:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.402282:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827891.402286:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:1.0:1787827891.402288:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827891.402296:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476760704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827891.402357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9a78a6775680 x1874671476760192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827891.402383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476760192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787827891.402396:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827891.402404:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827891.402411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827891.402417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476760192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827891.402426:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827891.402452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827891.402468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9a78a6776d00 x1874671476760320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827891.402481:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476760320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827891.402490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827891.402494:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827891.402498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827891.402529:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827891.402553:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827891.402566:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9a78a67761c0 x1874671476760448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827891.402579:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476760448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827891.402588:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827891.402592:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827891.402596:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827891.402603:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827891.402624:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827891.402635:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9a78a6774780 x1874671476760576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827891.402648:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476760576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827891.402659:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827891.402663:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827891.402667:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827891.402673:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827891.402694:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827891.402707:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9a78a67770c0 x1874671476760704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827891.402720:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476760704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827891.402731:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827891.402735:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827891.402739:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827892.425991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827893.450065:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827894.473944:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827895.498052:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827896.521898:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827896.521909:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827896.521913:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.521924:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2264 00000100:00080000:3.0:1787827896.521935:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.521943:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827896.522000:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476760832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827896.522028:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827896.522032:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827896.522033:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.522037:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2264 00000100:00080000:3.0:1787827896.522040:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.522044:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827896.522059:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827896.522069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476760960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827896.522078:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827896.522080:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476760832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827896.522081:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827896.522085:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.522092:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2264 00000100:00080000:3.0:1787827896.522095:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.522097:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827896.522101:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827896.522113:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476761088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827896.522121:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827896.522124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827896.522126:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.522129:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2264 00000100:00080000:3.0:1787827896.522149:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.522153:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827896.522158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827896.522166:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827896.522172:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476761216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827896.522185:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827896.522189:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827896.522191:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.522196:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2264 00000100:00080000:3.0:1787827896.522201:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.522205:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827896.522211:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827896.522221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476761344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827896.522248:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9a78a6776940 x1874671476760832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827896.522268:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476760832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827896.522278:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827896.522283:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827896.522288:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476760832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827896.522319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760960:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787827896.522322:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:1.0:1787827896.522341:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827896.522353:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9a78a6776580 x1874671476760960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827896.522361:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476760960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827896.522367:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827896.522369:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827896.522371:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476760960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827896.522375:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827896.522385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827896.522391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9a78a6774b40 x1874671476761088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827896.522403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476761088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827896.522408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827896.522410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827896.522412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827896.522416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827896.522427:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827896.522433:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9a78a67752c0 x1874671476761216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827896.522441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476761216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827896.522447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827896.522449:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827896.522451:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827896.522455:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827896.522464:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827896.522470:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9a78a6775a40 x1874671476761344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827896.522477:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476761344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827896.522488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827896.522490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827896.522493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827897.546022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827898.569993:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827899.593996:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827900.618020:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827901.641944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827901.641955:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827901.641959:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827901.641970:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2269 00000100:00080000:3.0:1787827901.641982:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827901.641989:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827901.642032:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476761472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827901.642061:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827901.642067:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827901.642068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827901.642074:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2269 00000100:00080000:3.0:1787827901.642077:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827901.642080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827901.642090:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827901.642099:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476761600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827901.642107:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827901.642110:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827901.642111:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827901.642114:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2269 00000100:00080000:3.0:1787827901.642117:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827901.642119:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827901.642122:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827901.642132:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476761472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827901.642134:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476761728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827901.642149:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827901.642152:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827901.642153:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827901.642156:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2269 00000100:00080000:3.0:1787827901.642158:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827901.642160:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827901.642165:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827901.642172:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476761856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827901.642179:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827901.642182:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827901.642183:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827901.642186:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2269 00000100:00080000:3.0:1787827901.642202:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827901.642205:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827901.642208:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:1.0:1787827901.642215:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827901.642218:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476761984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827901.642272:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9a78a6777480 x1874671476761472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.642289:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476761472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.642299:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.642303:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827901.642306:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476761472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827901.642313:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761600:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787827901.642315:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:1.0:1787827901.642339:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827901.642354:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9a78a6774000 x1874671476761600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.642368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476761600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.642378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.642382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827901.642386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827901.642429:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827901.642453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827901.642466:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9a78a6775a40 x1874671476761728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.642480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476761728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.642494:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.642497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827901.642499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827901.642504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827901.642521:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827901.642538:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9a78a67752c0 x1874671476761856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.642562:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476761856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.642579:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.642583:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827901.642587:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827901.642594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827901.642617:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827901.642637:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9a78a6774b40 x1874671476761984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.642651:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476761984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.642662:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.642665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827901.642669:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476761984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827902.666051:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827903.690002:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827904.714063:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827905.738017:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827906.761980:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827906.761992:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827906.761997:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827906.762009:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2274 00000100:00080000:0.0:1787827906.762021:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827906.762028:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827906.762052:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827906.762071:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476762112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827906.762096:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827906.762099:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827906.762100:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827906.762103:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2274 00000100:00080000:0.0:1787827906.762106:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827906.762109:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827906.762119:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827906.762131:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476762240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827906.762136:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476762112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827906.762152:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827906.762156:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827906.762159:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827906.762164:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2274 00000100:00080000:0.0:1787827906.762169:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827906.762173:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827906.762179:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827906.762198:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476762368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827906.762209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827906.762213:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827906.762215:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827906.762220:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2274 00000100:00080000:0.0:1787827906.762225:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787827906.762231:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827906.762234:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827906.762241:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827906.762253:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476762496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827906.762260:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827906.762263:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827906.762265:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827906.762268:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2274 00000100:00080000:0.0:1787827906.762271:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827906.762273:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827906.762276:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827906.762284:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476762624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827906.762312:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827906/real 1787827906] req@ffff9a7890a3c3c0 x1874671476762112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827906.762332:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476762112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827906.762342:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827906.762345:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827906.762349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476762112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827906.762373:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827906.762392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827906.762402:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827906/real 1787827906] req@ffff9a7890a3c000 x1874671476762240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827906.762410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476762240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827906.762421:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827906.762423:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827906.762426:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827906.762429:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827906.762440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827906.762446:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827906/real 1787827906] req@ffff9a7890a3f480 x1874671476762368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827906.762453:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476762368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827906.762459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827906.762461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827906.762463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827906.762466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827906.762477:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827906.762483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827906/real 1787827906] req@ffff9a7890a3de00 x1874671476762496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827906.762491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476762496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827906.762497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827906.762499:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827906.762502:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827906.762505:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827906.762514:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827906.762520:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827906/real 1787827906] req@ffff9a7890a3da40 x1874671476762624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827906.762527:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476762624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827906.762533:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827906.762534:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827906.762536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827907.785991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827908.810008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827909.833983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827910.857997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827911.881940:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827911.881948:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827911.881951:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827911.881961:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2279 00000100:00080000:0.0:1787827911.881969:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827911.881976:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827911.881978:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827911.882008:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476762752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827911.882031:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827911.882034:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827911.882035:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827911.882038:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2279 00000100:00080000:0.0:1787827911.882040:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827911.882042:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827911.882050:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827911.882060:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476762880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827911.882067:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827911.882069:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827911.882071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827911.882073:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2279 00000100:00080000:0.0:1787827911.882075:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827911.882080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827911.882083:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476762752:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787827911.882084:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827911.882096:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476763008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827911.882108:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827911.882110:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827911.882112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827911.882115:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2279 00000100:00080000:0.0:1787827911.882117:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827911.882119:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827911.882123:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827911.882130:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476763136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827911.882136:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827911.882138:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827911.882139:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1787827911.882144:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827911.882147:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2279 00000100:00080000:0.0:1787827911.882151:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827911.882154:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827911.882158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827911.882166:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476763264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827911.882191:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827911/real 1787827911] req@ffff9a7890a3fc00 x1874671476762752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827911.882206:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476762752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827911.882214:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827911.882217:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827911.882220:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476762752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827911.882225:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827911.882242:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827911.882251:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827911/real 1787827911] req@ffff9a7890a3da40 x1874671476762880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827911.882264:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476762880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827911.882273:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827911.882276:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827911.882280:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476762880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827911.882305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827911.882324:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827911.882332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827911/real 1787827911] req@ffff9a7890a3de00 x1874671476763008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827911.882341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476763008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827911.882347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827911.882349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827911.882351:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827911.882355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827911.882366:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827911.882372:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827911/real 1787827911] req@ffff9a7890a3f480 x1874671476763136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827911.882384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476763136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827911.882390:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827911.882391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827911.882393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827911.882397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827911.882406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827911.882412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827911/real 1787827911] req@ffff9a7890a3c000 x1874671476763264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827911.882418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476763264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827911.882425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827911.882426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827911.882428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827912.905983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827913.929975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827914.954011:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827915.977979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827917.001937:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.001946:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827917.001950:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.001959:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2284 00000100:00080000:1.0:1787827917.001968:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827917.001974:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827917.002005:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476763392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827917.002014:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827917.002027:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.002030:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827917.002032:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.002035:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2284 00000100:00080000:1.0:1787827917.002038:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827917.002041:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827917.002048:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827917.002059:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476763392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827917.002059:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476763520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827917.002073:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.002076:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827917.002077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.002080:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2284 00000100:00080000:1.0:1787827917.002082:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827917.002084:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827917.002088:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827917.002096:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476763648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827917.002103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.002105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827917.002107:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.002120:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2284 00000800:00000100:0.0:1787827917.002121:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827917.002125:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827917.002128:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827917.002134:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827917.002143:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476763776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827917.002150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827917.002153:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827917.002154:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827917.002157:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2284 00000100:00080000:1.0:1787827917.002160:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827917.002164:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9a7890b1e940 x1874671476763392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827917.002165:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827917.002172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827917.002185:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476763904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827917.002195:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476763392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827917.002204:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827917.002207:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827917.002210:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476763392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827917.002229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827917.002244:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827917.002253:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9a7890b1f0c0 x1874671476763520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827917.002261:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476763520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827917.002267:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827917.002269:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827917.002272:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827917.002278:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827917.002298:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827917.002309:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9a7890b1f480 x1874671476763648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827917.002322:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476763648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827917.002336:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827917.002340:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827917.002344:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827917.002351:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827917.002368:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827917.002380:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9a7890b1f840 x1874671476763776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827917.002391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476763776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827917.002401:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827917.002404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827917.002406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827917.002410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827917.002424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827917.002431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9a7890b1fc00 x1874671476763904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827917.002439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476763904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827917.002445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827917.002447:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827917.002450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476763904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827918.025975:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827919.050088:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827920.073993:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827921.098037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827922.121951:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827922.121962:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827922.121967:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827922.121978:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2289 00010000:00010000:1.0:1787827922.121988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827922.121991:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827922.122000:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827922.122044:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476764032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827922.122069:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827922.122072:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827922.122074:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827922.122077:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2289 00000100:00080000:0.0:1787827922.122080:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827922.122084:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827922.122094:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827922.122099:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476764032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827922.122109:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476764160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827922.122118:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827922.122122:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827922.122135:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827922.122139:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2289 00000100:00080000:0.0:1787827922.122143:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827922.122149:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827922.122155:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827922.122171:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476764288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827922.122182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827922.122187:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827922.122190:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827922.122194:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2289 00000100:00080000:0.0:1787827922.122198:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827922.122202:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827922.122226:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827922.122240:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476764416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827922.122249:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827922.122252:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764160:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827922.122253:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827922.122257:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827922.122262:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2289 00000100:00080000:0.0:1787827922.122265:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827922.122268:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827922.122284:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827922.122299:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476764544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827922.122302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827922.122334:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827922.122362:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827923.146216:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827924.170159:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827925.194243:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787827925.194287:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00010000:00010000:1.0:1787827925.194303:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:0.0:1787827925.194309:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827925.194358:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787827925.194377:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827925.194386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827925] req@ffff9a7890a3c000 x1874671476764160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:0.0:1787827925.194394:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787827925.194424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476764160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:0.0:1787827925.194434:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00080000:1.0:1787827925.194441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827925.194448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827925.194454:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827925.194460:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827925] req@ffff9a7890a3f480 x1874671476764288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827925.194475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476764288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827925.194486:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827925.194490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827925.194495:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827925.194503:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827925] req@ffff9a7890a3c3c0 x1874671476764032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827925.194522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476764032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827925.194557:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827925.194562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827925.194567:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476764032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827925.194596:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827925] req@ffff9a7890a3de00 x1874671476764416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827925.194614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476764416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827925.194627:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827925.194632:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827925.194636:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827925.194645:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827925] req@ffff9a7890a3da40 x1874671476764544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827925.194662:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476764544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827925.194675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827925.194679:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827925.194685:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827926.218267:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827927.242023:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827927.242037:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827927.242043:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827927.242058:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2294 00000100:00080000:0.0:1787827927.242085:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827927.242097:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827927.242155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476764672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827927.242197:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827927.242202:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827927.242205:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827927.242209:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827927.242212:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2294 00000100:00080000:0.0:1787827927.242220:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827927.242236:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827927.242251:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827927.242267:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476764800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827927.242281:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827927.242286:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827927.242289:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827927.242298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476764672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827927.242300:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2294 00000100:00080000:0.0:1787827927.242308:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827927.242316:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827927.242324:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827927.242346:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476764928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827927.242364:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827927.242371:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827927.242375:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827927.242383:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2294 00000100:00080000:0.0:1787827927.242405:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827927.242413:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827927.242422:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787827927.242426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827927.242454:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476765056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827927.242473:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827927.242479:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827927.242483:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827927.242492:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2294 00000100:00080000:0.0:1787827927.242500:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827927.242509:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827927.242520:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827927.242524:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9a7890a3fc00 x1874671476764672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827927.242545:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476765184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827927.242597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476764672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.242632:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.242641:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827927.242649:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476764672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827927.242693:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827927.242735:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827927.242803:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9a7890a3da40 x1874671476764800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827927.242826:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476764800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.242840:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.242886:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827927.242893:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827927.242903:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827927.242939:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827927.242961:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9a7890a3de00 x1874671476764928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827927.242979:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476764928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.242992:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.242997:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827927.243003:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476764928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827927.243014:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827927.243047:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827927.243066:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9a7890a3c3c0 x1874671476765056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827927.243086:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476765056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.243102:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.243106:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827927.243111:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827927.243121:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827927.243151:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827927.243171:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9a7890a3f480 x1874671476765184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827927.243191:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476765184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.243207:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.243212:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827927.243217:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827928.266319:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827929.289960:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827930.314059:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827931.338022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827932.361954:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827932.361968:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827932.361973:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827932.361983:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2299 00000100:00080000:0.0:1787827932.361992:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827932.362000:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827932.362038:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476765312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827932.362069:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827932.362074:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827932.362076:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827932.362076:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827932.362082:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2299 00000100:00080000:0.0:1787827932.362085:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827932.362090:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827932.362101:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827932.362116:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476765440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827932.362125:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827932.362128:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827932.362129:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827932.362133:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2299 00000100:00080000:0.0:1787827932.362136:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827932.362138:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827932.362141:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827932.362148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476765568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827932.362157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827932.362159:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827932.362161:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827932.362164:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2299 00000100:00080000:0.0:1787827932.362166:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827932.362168:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827932.362172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827932.362187:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476765696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827932.362190:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476765312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827932.362195:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827932.362214:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827932.362216:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827932.362221:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2299 00000100:00080000:0.0:1787827932.362224:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827932.362226:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827932.362229:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827932.362241:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476765824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827932.362338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827932.362450:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff9a7890a3c000 x1874671476765312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827932.362491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476765312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827932.362508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827932.362514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827932.362520:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476765312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827932.362532:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827932.362568:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827932.362586:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff9a7890a3f480 x1874671476765440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827932.362602:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476765440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827932.362612:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827932.362617:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827932.362621:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827932.362689:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827932.362758:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827932.362778:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff9a7890a3c3c0 x1874671476765568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827932.362803:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476765568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827932.362815:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827932.362819:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827932.362823:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827932.362870:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827932.362900:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827932.362921:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff9a7890a3de00 x1874671476765696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827932.362945:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476765696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827932.362965:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827932.362972:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827932.362979:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827932.362992:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827932.363029:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827932.363057:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827932/real 1787827932] req@ffff9a7890a3da40 x1874671476765824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827932.363072:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476765824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827932.363093:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827932.363097:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827932.363101:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476765824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827933.385963:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827934.410047:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827935.434017:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827936.458019:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827937.481965:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827937.481980:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827937.481986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827937.482008:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2304 00000100:00080000:0.0:1787827937.482021:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827937.482030:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827937.482036:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827937.482086:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476765952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827937.482126:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827937.482132:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827937.482136:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827937.482144:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2304 00000100:00080000:0.0:1787827937.482150:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827937.482158:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827937.482167:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476765952:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787827937.482170:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827937.482195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476766080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827937.482209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827937.482213:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827937.482215:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827937.482232:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2304 00000100:00080000:0.0:1787827937.482255:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827937.482262:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:1.0:1787827937.482266:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787827937.482275:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827937.482301:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476766208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827937.482319:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827937.482325:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1787827937.482329:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827937/real 1787827937] req@ffff9a7890a3fc00 x1874671476765952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827937.482331:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827937.482344:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2304 00000100:00080000:0.0:1787827937.482352:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827937.482360:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827937.482368:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827937.482369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476765952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827937.482387:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787827937.482388:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476766336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827937.482397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827937.482409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476765952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787827937.482421:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827937.482427:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827937.482431:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827937.482440:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2304 00000100:00100000:1.0:1787827937.482445:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787827937.482449:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827937.482461:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827937.482471:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:1.0:1787827937.482478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827937.482494:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476766464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827937.482497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827937/real 1787827937] req@ffff9a7890a3da40 x1874671476766080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827937.482554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476766080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827937.482565:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827937.482570:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827937.482574:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827937.482581:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827937.482607:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827937.482622:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827937/real 1787827937] req@ffff9a7890a3de00 x1874671476766208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827937.482637:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476766208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827937.482647:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827937.482654:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827937.482663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827937.482694:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827937.482741:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827937.482760:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827937/real 1787827937] req@ffff9a7890a3c3c0 x1874671476766336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827937.482780:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476766336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827937.482790:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827937.482794:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827937.482797:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827937.482816:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827937.482876:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827937.482892:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827937/real 1787827937] req@ffff9a7890a3f480 x1874671476766464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827937.482915:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476766464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827937.482924:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827937.482927:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827937.482931:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827938.505974:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827939.530106:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827940.554012:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827941.578004:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827942.602027:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827942.602041:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827942.602050:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827942.602066:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2310 00000100:00080000:0.0:1787827942.602083:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827942.602093:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827942.602131:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787827942.602156:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476766592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827942.602194:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827942.602198:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827942.602201:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827942.602205:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2310 00000100:00080000:0.0:1787827942.602209:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827942.602213:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827942.602237:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827942.602255:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476766720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827942.602267:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827942.602271:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827942.602273:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827942.602278:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2310 00000100:00080000:0.0:1787827942.602281:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827942.602285:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827942.602290:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827942.602297:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476766592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827942.602316:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476766848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827942.602338:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827942.602342:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827942.602344:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827942.602348:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2310 00000100:00080000:0.0:1787827942.602352:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827942.602355:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827942.602362:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827942.602375:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476766976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827942.602386:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827942.602390:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827942.602391:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827942.602395:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2310 00000100:00080000:0.0:1787827942.602400:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827942.602403:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827942.602408:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:1.0:1787827942.602418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827942.602424:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476767104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827942.602538:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827942/real 1787827942] req@ffff9a7890a3c000 x1874671476766592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827942.602566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476766592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827942.602581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827942.602585:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827942.602590:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476766592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827942.602600:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827942.602683:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827942.602705:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827942/real 1787827942] req@ffff9a7890a3f480 x1874671476766720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827942.602730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476766720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827942.602741:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827942.602745:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827942.602750:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827942.602786:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787827942.602812:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827942.602825:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827942/real 1787827942] req@ffff9a7890a3c3c0 x1874671476766848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827942.602884:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476766848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827942.602905:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827942.602909:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827942.602913:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787827942.602922:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827942.602959:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827942.602991:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827942/real 1787827942] req@ffff9a7890a3de00 x1874671476766976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827942.603015:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476766976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827942.603056:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827942.603061:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827942.603065:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476766976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787827942.603073:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787827942.603101:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827942.603115:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827942/real 1787827942] req@ffff9a7890a3da40 x1874671476767104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827942.603129:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476767104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827942.603158:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827942.603163:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787827942.603167:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827943.626179:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827944.650062:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827945.674105:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827946.697996:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827947.721940:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827947.721948:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827947.721953:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827947.721964:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2315 00000100:00080000:1.0:1787827947.721976:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827947.721979:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827947.721985:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827947.722024:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476767232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827947.722051:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827947.722054:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827947.722055:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827947.722058:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2315 00000100:00080000:1.0:1787827947.722060:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827947.722063:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827947.722071:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827947.722082:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476767360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827947.722090:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827947.722093:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476767232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827947.722094:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827947.722097:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827947.722104:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2315 00000100:00080000:1.0:1787827947.722106:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827947.722108:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827947.722111:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827947.722120:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476767488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827947.722139:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827947.722141:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827947.722143:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827947.722145:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2315 00000100:00080000:1.0:1787827947.722157:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827947.722160:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827947.722163:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787827947.722168:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827947.722175:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476767616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827947.722183:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827947.722185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827947.722186:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827947.722189:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2315 00000100:00080000:1.0:1787827947.722191:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827947.722193:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827947.722197:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827947.722205:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476767744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827947.722214:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827947/real 1787827947] req@ffff9a7890b1c000 x1874671476767232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827947.722233:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476767232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827947.722242:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827947.722245:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827947.722248:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476767232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827947.722265:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827947.722279:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827947.722286:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827947/real 1787827947] req@ffff9a7890b1fc00 x1874671476767360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827947.722294:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476767360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827947.722300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827947.722301:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827947.722304:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827947.722307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827947.722317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827947.722322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827947/real 1787827947] req@ffff9a7890b1f840 x1874671476767488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827947.722330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476767488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827947.722339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827947.722341:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827947.722343:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827947.722346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827947.722356:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827947.722361:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827947/real 1787827947] req@ffff9a7890b1f480 x1874671476767616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827947.722368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476767616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827947.722375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827947.722377:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827947.722378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827947.722381:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827947.722390:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827947.722395:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827947/real 1787827947] req@ffff9a7890b1f0c0 x1874671476767744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827947.722403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476767744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827947.722409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827947.722410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827947.722413:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476767744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827948.745956:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827949.769997:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827950.794000:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827951.817957:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827952.841913:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827952.841925:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827952.841930:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827952.841943:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2320 00000100:00080000:1.0:1787827952.841960:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827952.841970:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827952.842013:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827952.842032:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476767872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827952.842056:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827952.842061:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827952.842062:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827952.842067:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2320 00000100:00080000:1.0:1787827952.842073:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827952.842076:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827952.842087:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827952.842102:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476768000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827952.842113:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827952.842119:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787827952.842123:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476767872:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827952.842123:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827952.842135:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2320 00000100:00080000:1.0:1787827952.842147:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827952.842151:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827952.842155:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827952.842170:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476768128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827952.842178:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827952.842182:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827952.842183:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827952.842189:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2320 00000100:00080000:1.0:1787827952.842192:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827952.842195:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827952.842200:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827952.842209:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476768256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827952.842217:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827952.842220:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827952.842221:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827952.842225:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2320 00000100:00080000:1.0:1787827952.842227:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827952.842236:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827952.842238:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827952.842248:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827952.842262:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476768384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827952.842322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827952/real 1787827952] req@ffff9a7890b1e940 x1874671476767872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827952.842349:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476767872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827952.842370:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827952.842378:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827952.842387:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476767872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827952.842428:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827952.842473:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827952.842493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827952/real 1787827952] req@ffff9a7890b1da40 x1874671476768000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827952.842510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476768000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827952.842521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827952.842525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827952.842530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827952.842539:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827952.842591:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827952.842610:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827952/real 1787827952] req@ffff9a7890b1e580 x1874671476768128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827952.842626:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476768128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827952.842637:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827952.842641:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827952.842645:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827952.842653:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827952.842678:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827952.842690:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827952/real 1787827952] req@ffff9a7890b1de00 x1874671476768256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827952.842704:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476768256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827952.842716:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827952.842720:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827952.842723:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827952.842731:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827952.842749:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827952.842760:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827952/real 1787827952] req@ffff9a7890b1c3c0 x1874671476768384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827952.842774:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476768384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827952.842786:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827952.842789:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827952.842793:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827953.865965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827954.890000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827955.913992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827956.937983:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827957.962006:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827957.962023:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827957.962031:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827957.962047:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2325 00000100:00080000:1.0:1787827957.962076:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827957.962089:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827957.962107:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787827957.962158:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476768512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827957.962215:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827957.962223:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827957.962227:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827957.962235:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2325 00000100:00080000:1.0:1787827957.962244:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827957.962253:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827957.962268:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827957.962272:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476768512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787827957.962316:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476768640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827957.962369:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827957.962375:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827957.962379:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827957.962386:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2325 00000100:00080000:1.0:1787827957.962404:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827957.962411:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827957.962438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827957.962441:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827957.962470:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476768768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827957.962490:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827957.962497:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827957.962502:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827957.962512:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827957/real 1787827957] req@ffff9a7890b1d680 x1874671476768512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827957.962513:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2325 00000100:00080000:1.0:1787827957.962578:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827957.962590:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827957.962598:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476768512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787827957.962602:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:0.0:1787827957.962624:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787827957.962629:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476768896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827957.962633:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827957.962644:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476768512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827957.962678:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827957.962685:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827957.962689:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827957.962690:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768640:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787827957.962706:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2325 00000100:00080000:1.0:1787827957.962716:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827957.962724:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827957.962733:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787827957.962735:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827957.962759:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827957/real 1787827957] req@ffff9a7890b1c3c0 x1874671476768640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827957.962764:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476769024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827957.962814:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476768640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827957.962894:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827957.962899:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827957.962904:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827957.962913:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827957.962943:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827957.962975:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827957/real 1787827957] req@ffff9a7890b1de00 x1874671476768768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827957.962989:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476768768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827957.962998:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827957.963002:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827957.963005:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827957.963011:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827957.963030:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827957.963039:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827957/real 1787827957] req@ffff9a7890b1e580 x1874671476768896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827957.963051:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476768896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827957.963061:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827957.963064:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827957.963067:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476768896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827957.963085:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827957.963105:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827957.963128:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827957/real 1787827957] req@ffff9a7890b1da40 x1874671476769024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827957.963151:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476769024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827982 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827957.963168:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827957.963174:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827957.963180:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827958.985976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827960.010089:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827961.034003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827962.057986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827963.082045:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827963.082061:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827963.082069:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827963.082091:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2330 00000100:00080000:1.0:1787827963.082110:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827963.082122:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827963.082186:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476769152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827963.082199:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827963.082233:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827963.082240:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827963.082243:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827963.082248:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2330 00000100:00080000:1.0:1787827963.082253:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827963.082258:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827963.082286:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827963.082302:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476769280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827963.082319:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827963.082326:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827963.082330:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787827963.082330:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476769152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827963.082366:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2330 00000100:00080000:1.0:1787827963.082371:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827963.082376:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827963.082383:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827963.082402:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476769408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827963.082415:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827963.082420:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827963.082424:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827963.082429:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2330 00000100:00080000:1.0:1787827963.082452:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827963.082457:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827963.082465:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827963.082476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787827963.082485:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476769536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827963.082558:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827963.082562:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827963.082565:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827963.082572:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2330 00000100:00080000:1.0:1787827963.082606:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827963.082612:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827963.082621:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787827963.082638:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476769664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827963.082682:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff9a7890b1e940 x1874671476769152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827963.082725:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476769152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827963.082761:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827963.082770:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827963.082794:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476769152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827963.082899:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827963.082959:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827963.082987:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff9a7890b1f0c0 x1874671476769280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827963.083012:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476769280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827963.083029:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827963.083037:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827963.083044:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827963.083057:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827963.083099:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827963.083133:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff9a7890b1f480 x1874671476769408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827963.083159:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476769408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827963.083177:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827963.083184:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827963.083191:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827963.083204:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827963.083260:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827963.083284:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff9a7890b1f840 x1874671476769536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827963.083323:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476769536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827963.083352:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827963.083358:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827963.083365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827963.083377:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827963.083415:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827963.083440:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827963/real 1787827963] req@ffff9a7890b1fc00 x1874671476769664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827963.083462:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476769664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827963.083481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827963.083514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827963.083523:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827964.106132:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827965.130025:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827966.154145:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827967.178108:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827968.202052:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827968.202068:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827968.202075:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827968.202091:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2335 00000100:00080000:3.0:1787827968.202109:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827968.202121:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787827968.202154:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787827968.202191:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476769792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827968.202240:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827968.202247:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827968.202251:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827968.202258:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2335 00000100:00080000:3.0:1787827968.202265:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827968.202273:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827968.202292:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827968.202319:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476769920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827968.202341:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827968.202350:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827968.202354:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827968.202364:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2335 00000100:00080000:3.0:1787827968.202371:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827968.202384:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827968.202387:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476769792:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787827968.202394:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827968.202416:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476770048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827968.202433:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827968.202438:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827968.202441:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827968.202447:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2335 00000100:00080000:3.0:1787827968.202476:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827968.202485:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827968.202495:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827968.202517:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476770176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827968.202533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787827968.202549:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827968.202558:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827968.202563:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827968.202573:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2335 00000100:00080000:3.0:1787827968.202583:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827968.202593:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827968.202606:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827968.202620:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9a78a6776580 x1874671476769792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827968.202632:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476770304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827968.202664:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476769792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827968.202685:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827968.202692:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827968.202698:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476769792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827968.202708:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827968.202737:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827968.202754:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9a78a6776940 x1874671476769920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827968.202769:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476769920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827968.202780:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827968.202784:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827968.202788:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476769920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827968.202881:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827968.202908:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827968.202923:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9a78a67770c0 x1874671476770048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827968.202937:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476770048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827968.202949:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827968.202953:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827968.202957:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827968.202964:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827968.202984:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827968.202995:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9a78a6774780 x1874671476770176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827968.203009:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476770176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827968.203021:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827968.203025:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827968.203029:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827968.203036:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827968.203054:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827968.203065:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827968/real 1787827968] req@ffff9a78a67761c0 x1874671476770304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827968.203078:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476770304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827968.203090:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827968.203093:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827968.203097:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827969.226087:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827970.250169:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827971.274082:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827972.298129:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827973.322018:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827973.322035:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827973.322043:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827973.322059:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2340 00010000:00010000:2.0:1787827973.322060:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827973.322076:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827973.322088:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827973.322143:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476770432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827973.322184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827973.322193:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827973.322196:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827973.322202:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2340 00000100:00080000:3.0:1787827973.322209:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827973.322218:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827973.322235:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827973.322266:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476770432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827973.322268:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476770560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827973.322303:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827973.322310:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827973.322315:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827973.322322:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2340 00000100:00080000:3.0:1787827973.322345:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827973.322354:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787827973.322363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787827973.322370:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827973.322399:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476770688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827973.322414:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827973.322442:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827973.322445:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827973.322458:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2340 00000100:00100000:0.0:1787827973.322460:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff9a78a6776d00 x1874671476770432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827973.322465:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827973.322481:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827973.322498:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827973.322501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476770432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827973.322520:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827973.322531:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827973.322534:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476770816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827973.322539:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476770432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787827973.322570:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827973.322576:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827973.322579:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827973.322587:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770560:192.168.203.140@tcp:38:kworker.0 00000100:00080000:3.0:1787827973.322589:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2340 00000100:00080000:3.0:1787827973.322603:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827973.322613:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827973.322623:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:0.0:1787827973.322641:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827973.322650:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476770944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827973.322666:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff9a78a6775680 x1874671476770560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827973.322691:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476770560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827973.322709:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827973.322717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827973.322724:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827973.322738:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827973.322779:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827973.322803:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff9a78a6777840 x1874671476770688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827973.322821:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476770688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827973.322888:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827973.322893:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827973.322897:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827973.322906:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827973.322937:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827973.322953:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff9a78a6774f00 x1874671476770816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827973.322977:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476770816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827973.322988:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827973.322992:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827973.322996:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827973.323019:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827973.323040:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827973.323052:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827973/real 1787827973] req@ffff9a78a67743c0 x1874671476770944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827973.323066:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476770944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787827998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827973.323077:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827973.323081:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827973.323084:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476770944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827974.346085:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827975.370106:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827976.394058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827977.418084:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827978.442010:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827978.442028:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827978.442037:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827978.442054:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2345 00000100:00080000:1.0:1787827978.442074:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827978.442080:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827978.442090:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787827978.442149:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476771072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827978.442187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827978.442192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827978.442194:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827978.442200:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2345 00000100:00080000:1.0:1787827978.442205:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827978.442210:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827978.442222:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827978.442241:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476771200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827978.442256:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476771072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787827978.442283:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827978.442287:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827978.442289:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827978.442296:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2345 00000100:00080000:1.0:1787827978.442300:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827978.442306:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827978.442313:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827978.442331:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476771328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827978.442344:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827978.442349:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787827978.442352:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827978.442357:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2345 00000800:00000100:0.0:1787827978.442365:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787827978.442367:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827978.442377:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827978.442411:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787827978.442435:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476771456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827978.442448:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827978.442452:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787827978.442456:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827978.442464:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2345 00000100:00080000:1.0:1787827978.442471:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827978.442476:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787827978.442483:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827978.442487:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff9a7890b1c000 x1874671476771072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827978.442503:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476771584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827978.442538:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476771072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827978.442556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827978.442563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827978.442581:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476771072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827978.442612:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827978.442646:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827978.442663:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff9a7890b1fc00 x1874671476771200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827978.442679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476771200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827978.442690:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827978.442694:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827978.442698:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827978.442706:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787827978.442725:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827978.442737:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff9a7890b1f840 x1874671476771328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827978.442751:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476771328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827978.442761:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827978.442764:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827978.442769:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827978.442776:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827978.442795:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827978.442806:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff9a7890b1f480 x1874671476771456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827978.442866:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476771456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827978.442879:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827978.442884:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827978.442889:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827978.442896:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787827978.442940:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787827978.442956:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827978/real 1787827978] req@ffff9a7890b1f0c0 x1874671476771584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827978.442971:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476771584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827978.442983:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827978.442987:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827978.442991:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827979.466065:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827980.490183:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827981.514080:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827982.538015:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827983.562036:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827983.562053:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827983.562060:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827983.562078:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2351 00000100:00080000:0.0:1787827983.562095:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827983.562100:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827983.562109:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827983.562174:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476771712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827983.562231:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827983.562239:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827983.562244:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827983.562254:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2351 00000100:00080000:0.0:1787827983.562261:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827983.562269:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827983.562287:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827983.562307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476771712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827983.562314:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476771840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827983.562339:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827983.562368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827983.562373:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827983.562380:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2351 00000100:00080000:0.0:1787827983.562386:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827983.562392:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827983.562400:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827983.562421:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476771968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827983.562436:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827983.562441:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827983.562445:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827983.562450:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2351 00000100:00080000:0.0:1787827983.562455:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827983.562459:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787827983.562469:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787827983.562470:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827983.562490:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476772096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827983.562504:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827983.562509:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827983.562511:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827983.562516:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2351 00000100:00080000:0.0:1787827983.562520:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827983.562525:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827983.562532:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827983.562551:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476772224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827983.562563:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827983/real 1787827983] req@ffff9a7890a3fc00 x1874671476771712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827983.562597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476771712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827983.562614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827983.562619:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827983.562625:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476771712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827983.562659:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827983.562691:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827983.562708:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827983/real 1787827983] req@ffff9a7890a3da40 x1874671476771840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827983.562723:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476771840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827983.562733:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827983.562737:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827983.562741:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827983.562748:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827983.562767:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827983.562778:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827983/real 1787827983] req@ffff9a7890a3de00 x1874671476771968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827983.562793:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476771968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827983.562803:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827983.562806:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827983.562810:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476771968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827983.562816:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827983.562884:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827983.562907:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827983/real 1787827983] req@ffff9a7890a3c3c0 x1874671476772096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827983.562944:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476772096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827983.562964:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827983.562971:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827983.562977:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827983.562988:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827983.563026:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827983.563050:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827983/real 1787827983] req@ffff9a7890a3f480 x1874671476772224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827983.563074:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476772224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827983.563094:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827983.563101:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827983.563108:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787827984.586051:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827985.610069:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827986.633969:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827987.657995:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827988.681964:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827988.681973:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827988.681978:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827988.681988:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2356 00010000:00010000:1.0:1787827988.681990:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827988.681999:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827988.682007:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787827988.682045:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476772352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827988.682077:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827988.682081:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827988.682082:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827988.682086:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2356 00000100:00080000:3.0:1787827988.682090:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827988.682093:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827988.682102:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827988.682123:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476772352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827988.682123:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476772480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827988.682150:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827988.682152:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827988.682153:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827988.682158:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2356 00000100:00080000:3.0:1787827988.682161:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827988.682163:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827988.682166:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827988.682177:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476772608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827988.682184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827988.682188:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787827988.682189:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827988.682193:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2356 00000100:00080000:3.0:1787827988.682204:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827988.682208:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827988.682213:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787827988.682221:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476772736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827988.682229:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827988.682231:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787827988.682232:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827988.682236:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2356 00000100:00080000:3.0:1787827988.682241:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827988.682246:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787827988.682251:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787827988.682272:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476772864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827988.682295:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827988.682363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827988.682396:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827988.682421:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827989.706082:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827990.730143:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827991.754168:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827991.754317:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787827991.754394:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827991.754420:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827991.754498:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827991.754520:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827991.754537:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787827991.754558:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787827991.754595:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827988/real 1787827991] req@ffff9a78a6777c00 x1874671476772480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827991.754659:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476772480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827991.754683:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827991.754693:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827991.754702:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827991.754719:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827988/real 1787827991] req@ffff9a78a67743c0 x1874671476772608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827991.754747:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476772608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827991.754764:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827991.754771:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827991.754779:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787827991.754793:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827988/real 1787827991] req@ffff9a78a6774f00 x1874671476772736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827991.754930:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476772736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827991.754997:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827991.755007:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827991.755015:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827991.755038:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827988/real 1787827991] req@ffff9a78a6777840 x1874671476772864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827991.755066:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476772864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827991.755085:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827991.755091:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827991.755097:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476772864:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787827991.755112:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827988/real 1787827991] req@ffff9a78a6775e00 x1874671476772352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827991.755139:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476772352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827991.755154:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827991.755161:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787827991.755168:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476772352:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787827992.778145:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827993.802056:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827993.802095:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827993.802114:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827993.802151:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2361 00000100:00080000:0.0:1787827993.802166:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827993.802167:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827993.802176:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787827993.802229:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476772992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827993.802260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827993.802264:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827993.802266:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827993.802270:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2361 00000100:00080000:0.0:1787827993.802289:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827993.802296:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827993.802334:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827993.802356:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476772992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787827993.802358:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476773120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827993.802401:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827993.802407:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827993.802411:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827993.802420:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2361 00000100:00080000:0.0:1787827993.802429:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827993.802438:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827993.802462:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827993.802478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476773248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827993.802492:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787827993.802497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787827993.802499:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787827993.802505:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827993.802512:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2361 00000100:00080000:0.0:1787827993.802518:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827993.802524:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827993.802531:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787827993.802547:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476773376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827993.802559:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827993.802563:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787827993.802565:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827993.802571:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2361 00000100:00080000:0.0:1787827993.802575:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827993.802579:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827993/real 1787827993] req@ffff9a7890a3c000 x1874671476772992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827993.802584:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787827993.802594:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787827993.802613:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476773504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827993.802616:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476772992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827993.802647:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827993.802654:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827993.802661:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476772992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827993.802695:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827993.802726:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827993.802745:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827993/real 1787827993] req@ffff9a7890a3f480 x1874671476773120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827993.802760:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476773120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827993.802770:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827993.802774:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827993.802778:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827993.802784:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827993.802808:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827993.802822:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827993/real 1787827993] req@ffff9a7890a3c3c0 x1874671476773248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827993.802838:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476773248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827993.802855:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827993.802884:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827993.802888:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827993.802896:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827993.802922:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827993.802935:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827993/real 1787827993] req@ffff9a7890a3de00 x1874671476773376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827993.802948:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476773376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827993.802960:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827993.802964:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827993.802968:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827993.802975:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827993.802996:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827993.803010:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827993/real 1787827993] req@ffff9a7890a3da40 x1874671476773504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827993.803024:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476773504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827993.803035:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827993.803039:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827993.803045:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787827994.826059:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827995.850097:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827996.874081:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827997.898082:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827998.921988:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827998.922008:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827998.922016:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827998.922033:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2366 00000100:00080000:2.0:1787827998.922050:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827998.922059:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787827998.922132:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787827998.922146:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476773632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827998.922183:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827998.922188:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827998.922190:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827998.922194:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2366 00000100:00080000:2.0:1787827998.922198:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827998.922203:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827998.922218:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827998.922233:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476773760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827998.922262:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476773632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787827998.922274:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827998.922281:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827998.922283:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827998.922290:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2366 00000100:00080000:2.0:1787827998.922294:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827998.922298:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827998.922304:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827998.922321:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476773888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827998.922333:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827998.922338:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787827998.922340:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827998.922345:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2366 00000100:00080000:2.0:1787827998.922360:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827998.922364:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827998.922371:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787827998.922374:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787827998.922390:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476774016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827998.922404:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827998.922408:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787827998.922410:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827998.922414:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2366 00000100:00080000:2.0:1787827998.922418:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827998.922422:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787827998.922428:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787827998.922442:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476774144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827998.922455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827998/real 1787827998] req@ffff9a78841d9e00 x1874671476773632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827998.922484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476773632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827998.922509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827998.922514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827998.922519:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476773632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787827998.922574:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827998.922606:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827998.922625:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827998/real 1787827998] req@ffff9a78841db480 x1874671476773760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827998.922643:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476773760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827998.922655:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827998.922659:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827998.922664:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827998.922673:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787827998.922698:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827998.922714:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827998/real 1787827998] req@ffff9a78841d8000 x1874671476773888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827998.922729:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476773888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827998.922740:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827998.922743:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827998.922748:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476773888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787827998.922755:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827998.922777:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827998.922791:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827998/real 1787827998] req@ffff9a78841d83c0 x1874671476774016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827998.922806:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476774016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827998.922819:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827998.922823:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827998.922828:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787827998.922835:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787827998.922907:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787827998.922927:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827998/real 1787827998] req@ffff9a78841da580 x1874671476774144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827998.922948:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476774144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828023 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827998.922964:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827998.922968:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787827998.922972:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787827999.946093:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828000.969959:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828001.994055:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828003.018024:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828004.041927:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828004.041937:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828004.041941:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828004.041953:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2371 00000100:00080000:0.0:1787828004.041966:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828004.041973:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828004.041999:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828004.042017:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476774272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828004.042043:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828004.042046:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828004.042048:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828004.042054:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2371 00000100:00080000:0.0:1787828004.042058:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828004.042064:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828004.042077:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828004.042087:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476774272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828004.042097:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476774400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828004.042112:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828004.042116:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828004.042118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828004.042123:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2371 00000100:00080000:0.0:1787828004.042128:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828004.042133:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828004.042139:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828004.042152:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476774528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828004.042163:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828004.042166:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828004.042172:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828004.042174:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828004.042181:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2371 00000100:00080000:0.0:1787828004.042187:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828004.042192:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828004.042198:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828004.042226:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff9a7890a3fc00 x1874671476774272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828004.042232:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476774656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828004.042248:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476774272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828004.042250:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828004.042261:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828004.042265:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828004.042270:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828004.042274:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2371 00000100:00080000:3.0:1787828004.042276:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787828004.042282:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828004.042282:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476774272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828004.042291:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828004.042297:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828004.042310:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476774784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828004.042319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828004.042342:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828004.042353:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff9a7890a3da40 x1874671476774400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828004.042362:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476774400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828004.042368:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828004.042370:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828004.042372:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828004.042377:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828004.042387:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828004.042393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff9a7890a3de00 x1874671476774528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828004.042401:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476774528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828004.042406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828004.042408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828004.042410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828004.042413:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828004.042423:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828004.042428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff9a7890a3c3c0 x1874671476774656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828004.042435:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476774656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828004.042442:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828004.042444:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828004.042446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828004.042455:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828004.042464:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828004.042470:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828004/real 1787828004] req@ffff9a7890a3f480 x1874671476774784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828004.042478:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476774784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828004.042485:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828004.042486:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828004.042488:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476774784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828005.066041:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828006.090073:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828007.114083:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828008.138132:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828009.162030:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828009.162043:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828009.162051:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828009.162065:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2376 00000100:00080000:0.0:1787828009.162079:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828009.162089:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828009.162129:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828009.162145:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476774912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828009.162210:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828009.162216:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828009.162220:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828009.162226:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2376 00000100:00080000:0.0:1787828009.162232:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828009.162238:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828009.162253:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828009.162281:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476774912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828009.162282:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476775040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828009.162309:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828009.162313:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828009.162315:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828009.162321:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2376 00000100:00080000:0.0:1787828009.162325:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828009.162330:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828009.162337:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828009.162353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476775168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828009.162367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828009.162371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828009.162373:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828009.162379:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2376 00000800:00000100:3.0:1787828009.162388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828009.162391:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828009.162400:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828009.162408:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828009.162423:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476775296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828009.162435:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828009.162440:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828009.162443:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828009.162449:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2376 00000100:00080000:0.0:1787828009.162454:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828009.162465:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828009.162465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828009] req@ffff9a7890a3c000 x1874671476774912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828009.162478:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828009.162502:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476775424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828009.162512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476774912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828009.162536:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828009.162542:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828009.162549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476774912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828009.162579:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828009.162606:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828009.162621:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828009] req@ffff9a7890a3f480 x1874671476775040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828009.162637:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476775040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828009.162656:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828009.162660:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828009.162664:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828009.162671:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828009.162689:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828009.162700:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828009] req@ffff9a7890a3c3c0 x1874671476775168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828009.162729:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476775168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828009.162740:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828009.162744:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828009.162748:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828009.162755:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828009.162776:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828009.162788:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828009] req@ffff9a7890a3de00 x1874671476775296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828009.162802:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476775296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828009.162814:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828009.162817:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828009.162822:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828009.162859:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828009.162882:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828009.162895:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828009/real 1787828009] req@ffff9a7890a3da40 x1874671476775424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828009.162910:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476775424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828009.162921:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828009.162925:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828009.162929:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828010.186142:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828011.209999:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828012.234103:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828013.258186:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828014.282006:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828014.282027:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828014.282038:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828014.282060:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2381 00000100:00080000:0.0:1787828014.282082:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828014.282097:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828014.282161:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828014.282206:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476775552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828014.282266:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828014.282278:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828014.282281:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828014.282289:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2381 00000100:00080000:0.0:1787828014.282296:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828014.282304:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828014.282321:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828014.282343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476775552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828014.282350:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476775680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828014.282381:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828014.282387:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828014.282391:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828014.282400:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2381 00000100:00080000:0.0:1787828014.282409:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828014.282419:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828014.282432:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828014.282459:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828014.282469:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476775808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828014.282498:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828014.282507:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828014.282513:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828014.282523:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2381 00000100:00080000:0.0:1787828014.282531:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828014.282540:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff9a7890a3fc00 x1874671476775552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828014.282547:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828014.282571:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828014.282587:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476775552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828014.282598:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476775936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828014.282652:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828014.282657:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828014.282664:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828014.282670:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828014.282672:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2381 00000100:00080000:3.0:1787828014.282691:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787828014.282705:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828014.282706:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476775552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828014.282723:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828014.282732:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828014.282758:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828014.282759:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476776064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828014.282824:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828014.282886:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff9a7890a3da40 x1874671476775680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828014.282903:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476775680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828014.282914:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828014.282919:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828014.282923:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828014.282931:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828014.282956:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828014.282969:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff9a7890a3de00 x1874671476775808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828014.282983:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476775808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828014.282993:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828014.282996:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828014.283000:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828014.283008:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828014.283026:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828014.283037:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff9a7890a3c3c0 x1874671476775936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828014.283051:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476775936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828014.283063:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828014.283067:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828014.283070:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476775936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828014.283091:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828014.283109:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828014.283121:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828014/real 1787828014] req@ffff9a7890a3f480 x1874671476776064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828014.283159:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476776064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828014.283171:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828014.283175:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828014.283179:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828015.306097:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828016.330077:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828017.354042:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828018.377950:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828019.401948:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828019.401956:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828019.401962:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828019.401964:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828019.401971:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2386 00000100:00080000:0.0:1787828019.401983:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828019.401990:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828019.402028:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476776192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828019.402056:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828019.402060:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828019.402062:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828019.402066:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2386 00000100:00080000:0.0:1787828019.402071:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828019.402077:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828019.402089:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828019.402118:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476776192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828019.402119:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476776320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828019.402145:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828019.402148:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828019.402149:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828019.402154:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2386 00000100:00080000:0.0:1787828019.402159:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828019.402162:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828019.402166:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828019.402175:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476776448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828019.402182:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828019.402185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828019.402186:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828019.402189:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2386 00000100:00080000:0.0:1787828019.402204:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828019.402208:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828019.402211:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828019.402216:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828019.402227:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476776576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828019.402240:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828019.402242:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828019.402244:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828019.402248:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2386 00000100:00080000:0.0:1787828019.402252:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828019.402257:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828019.402264:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828019.402274:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9a7890a3c000 x1874671476776192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828019.402279:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476776704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828019.402336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476776192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828019.402348:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828019.402352:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828019.402355:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476776192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828019.402380:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828019.402398:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828019.402408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9a7890a3f480 x1874671476776320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828019.402416:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476776320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828019.402422:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828019.402425:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828019.402427:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828019.402431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828019.402443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828019.402449:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9a7890a3c3c0 x1874671476776448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828019.402457:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476776448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828019.402462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828019.402464:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828019.402466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828019.402470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828019.402481:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828019.402491:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9a7890a3de00 x1874671476776576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828019.402504:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476776576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828019.402515:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828019.402519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828019.402522:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828019.402529:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828019.402551:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828019.402563:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828019/real 1787828019] req@ffff9a7890a3da40 x1874671476776704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828019.402572:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476776704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828019.402580:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828019.402583:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828019.402585:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828020.425958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828021.450042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828022.474034:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828023.497990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828024.521980:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828024.521993:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828024.521999:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828024.522014:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2392 00000100:00080000:2.0:1787828024.522029:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828024.522041:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828024.522118:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828024.522124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476776832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828024.522155:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828024.522158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828024.522160:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828024.522164:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2392 00000100:00080000:2.0:1787828024.522167:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828024.522171:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828024.522200:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828024.522225:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476776960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828024.522244:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476776832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828024.522260:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828024.522270:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828024.522273:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828024.522285:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2392 00000100:00080000:2.0:1787828024.522319:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828024.522324:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828024.522330:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828024.522346:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476777088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828024.522362:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828024.522366:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828024.522384:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828024.522385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828024.522400:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2392 00000100:00080000:2.0:1787828024.522409:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828024.522416:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828024.522423:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828024.522447:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476777216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828024.522459:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828024.522462:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828024.522464:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828024.522469:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2392 00000100:00080000:2.0:1787828024.522476:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828024.522479:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828024.522487:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828024.522504:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476777344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828024.522539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828024/real 1787828024] req@ffff9a78841d8b40 x1874671476776832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828024.522607:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476776832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828024.522627:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828024.522633:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828024.522639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476776832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828024.522686:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828024.522744:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828024.522776:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828024/real 1787828024] req@ffff9a78841db840 x1874671476776960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828024.522805:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476776960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828024.522822:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828024.522869:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828024.522875:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476776960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828024.522885:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828024.522929:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828024.522946:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828024/real 1787828024] req@ffff9a78841d92c0 x1874671476777088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828024.522971:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476777088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828024.522982:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828024.522986:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828024.522991:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828024.522999:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828024.523020:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828024.523035:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828024/real 1787828024] req@ffff9a78841d9680 x1874671476777216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828024.523049:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476777216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828024.523089:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828024.523094:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828024.523098:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828024.523117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828024.523144:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828024.523159:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828024/real 1787828024] req@ffff9a78841da940 x1874671476777344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828024.523185:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476777344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828024.523198:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828024.523202:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828024.523206:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828025.546150:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828026.570102:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828027.594139:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828028.618109:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828029.642042:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828029.642059:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828029.642068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828029.642079:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828029.642086:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2397 00000100:00080000:1.0:1787828029.642107:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828029.642118:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828029.642181:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476777472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828029.642220:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828029.642225:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828029.642227:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828029.642232:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2397 00000100:00080000:1.0:1787828029.642237:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828029.642243:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828029.642270:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828029.642287:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476777600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828029.642299:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476777472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828029.642308:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828029.642315:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828029.642317:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828029.642324:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2397 00000100:00080000:1.0:1787828029.642329:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828029.642334:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828029.642341:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828029.642360:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476777728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828029.642373:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828029.642379:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828029.642382:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828029.642386:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2397 00000100:00080000:1.0:1787828029.642401:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828029.642406:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828029.642413:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828029.642420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828029.642436:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476777856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828029.642450:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828029.642456:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828029.642460:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828029.642465:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2397 00000100:00080000:1.0:1787828029.642487:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828029.642493:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828029.642500:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828029.642520:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476777984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828029.642525:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828029/real 1787828029] req@ffff9a7890b1e940 x1874671476777472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828029.642597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476777472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828029.642626:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828029.642635:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828029.642643:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476777472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828029.642687:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828029.642742:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828029.642768:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828029/real 1787828029] req@ffff9a7890b1da40 x1874671476777600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828029.642791:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476777600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828029.642808:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828029.642815:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828029.642821:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828029.642872:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828029.642916:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828029.642940:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828029/real 1787828029] req@ffff9a7890b1e580 x1874671476777728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828029.642965:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476777728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828029.642997:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828029.643001:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828029.643006:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828029.643014:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828029.643070:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828029.643087:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828029/real 1787828029] req@ffff9a7890b1de00 x1874671476777856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828029.643103:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476777856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828029.643115:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828029.643119:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828029.643135:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828029.643144:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828029.643166:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828029.643179:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828029/real 1787828029] req@ffff9a7890b1c3c0 x1874671476777984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828029.643193:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476777984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828029.643205:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828029.643209:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828029.643214:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476777984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828030.666059:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828031.689971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828032.714180:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828033.738100:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828034.761927:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828034.761938:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828034.761944:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828034.761957:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2402 00010000:00010000:1.0:1787828034.761969:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828034.761970:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828034.761981:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828034.762046:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476778112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828034.762073:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828034.762076:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828034.762078:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828034.762081:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2402 00000100:00080000:2.0:1787828034.762088:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828034.762092:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828034.762103:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828034.762113:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476778112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828034.762116:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476778240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828034.762127:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828034.762130:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828034.762131:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828034.762135:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2402 00000100:00080000:2.0:1787828034.762138:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828034.762140:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828034.762143:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828034.762152:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476778368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828034.762164:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828034.762167:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828034.762168:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828034.762171:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2402 00000100:00080000:2.0:1787828034.762181:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828034.762184:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828034.762187:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828034.762194:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828034.762198:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476778496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828034.762205:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828034.762209:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828034.762211:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828034.762214:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2402 00000100:00080000:2.0:1787828034.762216:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828034.762220:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828034.762223:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828034.762231:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476778624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828034.762260:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828034/real 1787828034] req@ffff9a78841da940 x1874671476778112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828034.762278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476778112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828034.762287:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828034.762291:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828034.762294:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476778112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828034.762317:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828034.762332:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828034.762340:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828034/real 1787828034] req@ffff9a78841d9680 x1874671476778240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828034.762348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476778240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828034.762353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828034.762356:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828034.762358:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828034.762362:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828034.762372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828034.762379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828034/real 1787828034] req@ffff9a78841d92c0 x1874671476778368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828034.762386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476778368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828034.762392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828034.762393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828034.762395:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828034.762399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828034.762410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828034.762416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828034/real 1787828034] req@ffff9a78841db840 x1874671476778496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828034.762424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476778496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828034.762431:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828034.762433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828034.762435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828034.762438:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828034.762447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828034.762452:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828034/real 1787828034] req@ffff9a78841d8b40 x1874671476778624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828034.762460:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476778624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828034.762468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828034.762470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828034.762472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828035.786103:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828036.810145:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828037.834107:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828038.858093:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828039.882022:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828039.882039:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828039.882047:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828039.882065:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2407 00000100:00080000:2.0:1787828039.882083:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828039.882096:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828039.882131:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828039.882171:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476778752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828039.882223:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828039.882228:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828039.882230:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828039.882235:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2407 00000100:00080000:2.0:1787828039.882242:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828039.882256:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828039.882272:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828039.882291:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476778752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828039.882294:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476778880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828039.882332:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828039.882336:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828039.882338:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828039.882346:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2407 00000100:00080000:2.0:1787828039.882352:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828039.882356:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828039.882361:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828039.882376:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476779008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828039.882396:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828039.882402:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828039.882405:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828039.882413:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2407 00000100:00080000:2.0:1787828039.882421:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828039.882429:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828039.882440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828039.882449:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828039.882482:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476779136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828039.882500:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828039.882504:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828039.882507:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828039.882515:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2407 00000100:00080000:2.0:1787828039.882523:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828039.882530:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828039.882539:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828039.882546:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828039/real 1787828039] req@ffff9a78841da580 x1874671476778752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828039.882566:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476779264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828039.882585:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476778752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828039.882617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828039.882628:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828039.882636:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476778752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828039.882694:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828039.882733:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828039.882752:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828039/real 1787828039] req@ffff9a78841d83c0 x1874671476778880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828039.882769:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476778880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828039.882780:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828039.882785:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828039.882789:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476778880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828039.882799:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828039.882820:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828039.882867:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828039/real 1787828039] req@ffff9a78841d8000 x1874671476779008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828039.882884:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476779008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828039.882895:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828039.882899:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828039.882903:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828039.882912:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828039.882936:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828039.882949:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828039/real 1787828039] req@ffff9a78841db480 x1874671476779136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828039.882998:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476779136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828039.883028:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828039.883035:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828039.883042:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828039.883056:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828039.883092:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828039.883109:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828039/real 1787828039] req@ffff9a78841d9e00 x1874671476779264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828039.883124:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476779264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828039.883136:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828039.883140:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828039.883144:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828040.906081:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828041.930078:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828042.954208:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828043.978168:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828045.001966:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.001978:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828045.001984:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.001997:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2412 00000100:00080000:2.0:1787828045.002010:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828045.002020:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828045.002064:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476779392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828045.002095:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828045.002095:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.002105:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828045.002109:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.002116:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2412 00000100:00080000:2.0:1787828045.002120:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828045.002125:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828045.002135:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828045.002145:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476779392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828045.002148:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476779520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828045.002161:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.002164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828045.002166:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.002169:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2412 00000100:00080000:2.0:1787828045.002172:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828045.002175:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828045.002179:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828045.002188:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476779648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828045.002196:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.002199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828045.002201:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.002204:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2412 00000100:00080000:2.0:1787828045.002207:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828045.002209:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828045.002214:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828045.002220:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828045.002230:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476779776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828045.002238:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828045.002240:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828045.002242:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828045.002245:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2412 00000100:00080000:2.0:1787828045.002247:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828045.002250:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828045.002253:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828045.002263:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476779904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828045.002274:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9a78841dbc00 x1874671476779392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828045.002304:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476779392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828045.002330:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828045.002336:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828045.002339:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476779392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828045.002362:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828045.002384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828045.002395:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9a78841da1c0 x1874671476779520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828045.002406:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476779520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828045.002417:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828045.002420:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828045.002423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828045.002427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828045.002439:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828045.002446:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9a78841db0c0 x1874671476779648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828045.002454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476779648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828045.002460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828045.002462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828045.002464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828045.002468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828045.002480:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828045.002486:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9a78841d9a40 x1874671476779776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828045.002495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476779776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828045.002502:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828045.002504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828045.002507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828045.002511:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828045.002522:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828045.002528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828045/real 1787828045] req@ffff9a78841d8780 x1874671476779904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828045.002536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476779904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828045.002543:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828045.002545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828045.002549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476779904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828046.026036:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828047.050100:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828048.074054:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828049.098153:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828050.122027:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828050.122043:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828050.122051:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828050.122068:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2417 00000100:00080000:0.0:1787828050.122086:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828050.122098:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828050.122168:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828050.122169:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476780032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828050.122230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828050.122238:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828050.122244:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828050.122254:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2417 00000100:00080000:0.0:1787828050.122263:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828050.122274:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828050.122296:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828050.122298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476780032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828050.122332:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476780160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828050.122359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828050.122367:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828050.122372:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828050.122381:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2417 00000100:00080000:0.0:1787828050.122408:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828050.122413:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828050.122428:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828050.122443:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828050.122476:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476780288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828050.122499:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828050.122500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9a7890a3fc00 x1874671476780032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828050.122520:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828050.122531:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828050.122541:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2417 00000100:00100000:3.0:1787828050.122547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476780032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828050.122551:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828050.122568:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828050.122581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787828050.122582:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787828050.122595:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828050.122604:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476780032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828050.122612:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476780416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828050.122630:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828050.122635:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828050.122639:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828050.122644:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2417 00000100:00080000:0.0:1787828050.122648:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828050.122652:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828050.122658:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828050.122667:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828050.122676:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476780544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828050.122732:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828050.122756:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9a7890a3da40 x1874671476780160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828050.122773:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476780160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828050.122785:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828050.122790:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828050.122794:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828050.122802:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828050.122825:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828050.122877:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9a7890a3de00 x1874671476780288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828050.122930:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476780288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828050.122948:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828050.122955:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828050.122961:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828050.122974:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828050.123018:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828050.123043:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9a7890a3c3c0 x1874671476780416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828050.123060:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476780416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828050.123074:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828050.123078:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828050.123082:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828050.123106:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828050.123133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828050.123147:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828050/real 1787828050] req@ffff9a7890a3f480 x1874671476780544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828050.123161:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476780544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828050.123174:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828050.123177:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828050.123181:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828051.146054:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828052.170078:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828053.193956:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828054.218137:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828055.241987:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828055.241999:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828055.242005:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828055.242017:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2422 00000100:00080000:0.0:1787828055.242030:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828055.242039:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828055.242085:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476780672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828055.242132:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828055.242140:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828055.242145:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828055.242157:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2422 00000100:00080000:0.0:1787828055.242166:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828055.242178:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828055.242184:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00100000:0.0:1787828055.242195:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828055.242210:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476780672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828055.242222:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476780800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828055.242244:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828055.242249:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828055.242251:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828055.242256:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2422 00000100:00080000:0.0:1787828055.242272:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828055.242277:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828055.242282:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828055.242296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476780928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828055.242307:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828055.242313:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828055.242316:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828055.242326:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2422 00000100:00080000:0.0:1787828055.242334:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828055.242343:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828055.242355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780800:192.168.203.140@tcp:38:kworker.0 00000008:00100000:0.0:1787828055.242356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828055.242389:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476781056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828055.242406:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828055.242407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780928:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787828055.242422:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828055.242426:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828055.242432:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2422 00000100:00080000:0.0:1787828055.242439:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828055.242448:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828055.242455:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781056:192.168.203.140@tcp:8:kworker.0 00000008:00100000:0.0:1787828055.242456:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828055.242479:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476781184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828055.242504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828056.266302:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828057.290231:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828058.314349:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828058.314426:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787828058.314477:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828058.314496:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828058.314546:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828058.314556:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828058.314563:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828058.314573:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787828058.314646:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828058] req@ffff9a7890a3da40 x1874671476781184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828058.314709:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476781184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828058.314737:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828058.314747:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828058.314766:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781184:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828058.314778:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828058] req@ffff9a7890a3de00 x1874671476781056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828058.314799:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476781056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828058.314819:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828058.314911:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828058.314921:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828058.314936:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828058] req@ffff9a7890a3f480 x1874671476780800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828058.314964:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476780800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828058.314982:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828058.314991:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828058.314997:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828058.315011:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828058] req@ffff9a7890a3c3c0 x1874671476780928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828058.315033:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476780928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828058.315048:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828058.315053:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828058.315058:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476780928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828058.315069:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828055/real 1787828058] req@ffff9a7890a3c000 x1874671476780672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828058.315088:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476780672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828058.315101:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828058.315114:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828058.315120:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476780672:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787828059.338325:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828060.362050:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828060.362065:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828060.362071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828060.362114:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2427 00000100:00080000:1.0:1787828060.362130:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828060.362141:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828060.362192:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476781312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828060.362238:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828060.362246:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828060.362250:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828060.362259:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2427 00000100:00080000:1.0:1787828060.362276:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828060.362284:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828060.362298:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00100000:1.0:1787828060.362302:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828060.362307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476781312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828060.362339:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476781440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828060.362361:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828060.362366:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828060.362369:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828060.362376:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2427 00000100:00080000:1.0:1787828060.362412:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828060.362419:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828060.362444:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828060.362453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828060.362468:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476781568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828060.362486:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828060.362492:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828060.362496:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828060.362503:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2427 00000100:00080000:1.0:1787828060.362510:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828060.362516:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828060.362526:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828060.362549:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff9a7890b1d680 x1874671476781312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828060.362562:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476781696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828060.362581:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828060.362589:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787828060.362590:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476781312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828060.362593:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828060.362607:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2427 00000100:00080000:1.0:1787828060.362617:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828060.362620:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828060.362627:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828060.362634:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:1.0:1787828060.362641:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828060.362644:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476781312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828060.362666:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476781824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828060.362696:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828060.362734:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828060.362774:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff9a7890b1c3c0 x1874671476781440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828060.362794:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476781440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828060.362805:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828060.362809:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828060.362868:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828060.362879:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828060.362911:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828060.362929:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff9a7890b1de00 x1874671476781568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828060.362946:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476781568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828060.362958:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828060.362963:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828060.362968:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828060.362976:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828060.363008:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828060.363027:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff9a7890b1e580 x1874671476781696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828060.363277:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476781696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828060.363332:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828060.363344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828060.363351:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828060.363368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828060.363449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828060.363486:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828060/real 1787828060] req@ffff9a7890b1da40 x1874671476781824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828060.363507:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476781824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828060.363519:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828060.363523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828060.363527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476781824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828061.386198:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828062.410062:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828063.434087:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828064.458033:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828065.482020:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828065.482034:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828065.482040:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828065.482071:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2432 00000100:00080000:0.0:1787828065.482088:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828065.482099:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828065.482116:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828065.482148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476781952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828065.482184:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828065.482188:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828065.482191:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828065.482195:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2432 00000100:00080000:0.0:1787828065.482218:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828065.482225:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828065.482238:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828065.482255:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476782080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828065.482270:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828065.482281:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787828065.482284:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476781952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828065.482285:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828065.482309:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2432 00000100:00080000:0.0:1787828065.482314:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828065.482318:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828065.482326:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828065.482367:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476782208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828065.482380:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828065.482384:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828065.482388:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828065.482392:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2432 00000100:00080000:0.0:1787828065.482436:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828065.482450:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828065.482452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787828065.482473:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828065.482497:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476782336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828065.482532:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828065.482538:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828065.482543:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828065.482547:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828065/real 1787828065] req@ffff9a7890a3fc00 x1874671476781952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828065.482565:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2432 00000100:00080000:0.0:1787828065.482594:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828065.482609:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828065.482637:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476781952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828065.482639:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828065.482667:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476782464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828065.482678:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828065.482703:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828065.482713:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476781952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828065.482761:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828065.482873:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828065.482896:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828065/real 1787828065] req@ffff9a7890a3c000 x1874671476782080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828065.482914:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476782080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828065.482925:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828065.482930:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828065.482935:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828065.482944:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828065.482990:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828065.483004:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828065/real 1787828065] req@ffff9a7890a3c3c0 x1874671476782208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828065.483018:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476782208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828065.483029:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828065.483034:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828065.483040:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828065.483048:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828065.483070:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828065.483082:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828065/real 1787828065] req@ffff9a7890a3f480 x1874671476782336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828065.483096:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476782336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828065.483108:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828065.483112:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828065.483116:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828065.483122:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828065.483152:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828065.483174:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828065/real 1787828065] req@ffff9a7890a3de00 x1874671476782464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828065.483198:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476782464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828065.483220:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828065.483227:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828065.483234:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828066.506047:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828067.530131:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828068.554071:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828069.578241:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828070.601910:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828070.601924:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828070.601930:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828070.601944:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2438 00000100:00080000:0.0:1787828070.601957:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828070.601967:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828070.602009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828070.602020:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476782592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828070.602060:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828070.602064:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828070.602066:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828070.602069:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2438 00000100:00080000:0.0:1787828070.602073:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828070.602077:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828070.602091:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828070.602108:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476782592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828070.602109:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476782720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828070.602129:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828070.602139:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828070.602141:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828070.602146:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2438 00000100:00080000:0.0:1787828070.602151:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828070.602157:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828070.602164:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828070.602183:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476782848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828070.602198:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787828070.602205:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828070.602209:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828070.602215:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828070.602223:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2438 00000100:00080000:0.0:1787828070.602229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828070.602244:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828070.602251:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828070.602276:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476782976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828070.602287:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828070.602288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828070/real 1787828070] req@ffff9a7890a3da40 x1874671476782592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828070.602296:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828070.602302:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828070.602313:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2438 00000100:00080000:0.0:1787828070.602318:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828070.602322:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828070.602326:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476782592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828070.602329:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828070.602347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787828070.602351:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476783104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828070.602356:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828070.602368:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476782592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828070.602404:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828070.602424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828070.602435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828070/real 1787828070] req@ffff9a7890a3de00 x1874671476782720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828070.602466:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476782720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828070.602477:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828070.602481:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828070.602485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828070.602491:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828070.602512:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828070.602522:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828070/real 1787828070] req@ffff9a7890a3f480 x1874671476782848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828070.602536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476782848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828070.602569:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828070.602572:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828070.602575:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828070.602580:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828070.602595:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828070.602603:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828070/real 1787828070] req@ffff9a7890a3c3c0 x1874671476782976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828070.602612:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476782976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828070.602649:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828070.602653:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828070.602656:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476782976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828070.602663:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828070.602682:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828070.602692:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828070/real 1787828070] req@ffff9a7890a3c000 x1874671476783104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828070.602705:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476783104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828070.602716:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828070.602719:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828070.602723:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828071.625975:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828072.650054:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828073.674023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828074.698094:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828075.721994:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828075.722010:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828075.722018:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828075.722052:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2443 00000100:00080000:0.0:1787828075.722071:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828075.722084:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828075.722141:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828075.722154:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476783232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828075.722199:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828075.722205:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828075.722209:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828075.722218:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2443 00000100:00080000:0.0:1787828075.722227:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828075.722237:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828075.722257:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828075.722271:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476783232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828075.722294:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476783360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828075.722329:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828075.722335:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828075.722338:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828075.722345:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2443 00000100:00080000:0.0:1787828075.722378:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828075.722384:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828075.722392:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828075.722399:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828075.722419:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476783488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828075.722446:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828075.722452:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828075.722455:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828075.722461:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2443 00000100:00080000:0.0:1787828075.722466:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828075.722472:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828075.722480:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828075.722507:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476783616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828075.722523:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828075/real 1787828075] req@ffff9a7890a3fc00 x1874671476783232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828075.722527:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828075.722560:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828075.722569:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828075.722581:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2443 00000100:00080000:0.0:1787828075.722590:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828075.722591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476783232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828075.722608:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828075.722616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787828075.722621:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828075.722646:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828075.722659:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476783232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828075.722668:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476783744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828075.722758:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828075.722800:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828075.722820:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828075/real 1787828075] req@ffff9a7890a3c000 x1874671476783360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828075.722868:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476783360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828075.722880:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828075.722884:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828075.722888:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828075.722896:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828075.722921:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828075.722936:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828075/real 1787828075] req@ffff9a7890a3c3c0 x1874671476783488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828075.722950:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476783488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828075.722959:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828075.722963:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828075.722967:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828075.722974:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828075.722992:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828075.723002:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828075/real 1787828075] req@ffff9a7890a3f480 x1874671476783616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828075.723015:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476783616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828075.723027:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828075.723030:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828075.723034:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828075.723041:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828075.723058:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828075.723068:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828075/real 1787828075] req@ffff9a7890a3de00 x1874671476783744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828075.723081:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476783744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828075.723093:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828075.723097:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828075.723101:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476783744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828076.746066:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828077.770117:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828078.793993:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828079.818103:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828080.841999:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828080.842017:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828080.842026:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828080.842045:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2448 00000100:00080000:0.0:1787828080.842067:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828080.842082:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828080.842102:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828080.842155:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476783872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828080.842210:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828080.842219:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828080.842223:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828080.842232:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2448 00000100:00080000:0.0:1787828080.842240:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828080.842249:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828080.842267:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828080.842270:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476783872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828080.842294:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476784000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828080.842314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828080.842324:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828080.842327:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828080.842334:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2448 00000100:00080000:0.0:1787828080.842365:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828080.842374:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828080.842383:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828080.842383:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828080.842405:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476784128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828080.842423:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828080.842429:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828080.842432:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828080.842439:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2448 00000100:00080000:0.0:1787828080.842444:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828080.842450:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828080.842472:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828080.842489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828080/real 1787828080] req@ffff9a7890a3da40 x1874671476783872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828080.842492:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476784256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828080.842512:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828080.842519:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828080.842523:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828080.842527:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476783872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828080.842541:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2448 00000100:00080000:0.0:1787828080.842547:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828080.842555:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828080.842557:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787828080.842565:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828080.842565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828080.842573:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476783872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828080.842582:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476784384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828080.842628:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828080.842700:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828080.842724:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828080/real 1787828080] req@ffff9a7890a3de00 x1874671476784000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828080.842740:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476784000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828080.842751:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828080.842755:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828080.842760:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828080.842769:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828080.842796:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828080.842809:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828080/real 1787828080] req@ffff9a7890a3f480 x1874671476784128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828080.842824:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476784128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828080.842866:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828080.842870:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828080.842874:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828080.842882:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828080.842907:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828080.842921:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828080/real 1787828080] req@ffff9a7890a3c3c0 x1874671476784256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828080.842936:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476784256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828080.842949:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828080.842953:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828080.842956:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828080.842963:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828080.842981:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828080.842992:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828080/real 1787828080] req@ffff9a7890a3c000 x1874671476784384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828080.843007:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476784384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828080.843019:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828080.843022:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828080.843025:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828081.866055:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828082.890185:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828083.913998:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828084.938064:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828085.962040:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828085.962050:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828085.962055:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828085.962063:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828085.962067:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2453 00000100:00080000:1.0:1787828085.962077:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828085.962083:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828085.962115:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476784512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828085.962140:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828085.962143:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828085.962145:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828085.962147:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2453 00000100:00080000:1.0:1787828085.962150:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828085.962152:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828085.962160:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828085.962169:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476784640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828085.962176:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828085.962178:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828085.962180:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828085.962183:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2453 00000100:00080000:1.0:1787828085.962185:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828085.962188:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828085.962192:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828085.962199:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476784768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828085.962206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828085.962208:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828085.962210:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828085.962213:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2453 00000100:00080000:1.0:1787828085.962215:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828085.962217:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828085.962221:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828085.962229:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476784896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828085.962236:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828085.962239:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828085.962240:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828085.962250:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2453 00000100:00080000:1.0:1787828085.962254:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828085.962254:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476784512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828085.962269:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828085.962280:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828085.962299:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476785024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828085.962412:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828085.962507:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828085/real 1787828085] req@ffff9a7890b1e940 x1874671476784512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828085.962550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476784512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828085.962574:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828085.962584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828085.962594:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476784512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828085.962624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828085.962709:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828085.962739:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828085/real 1787828085] req@ffff9a7890b1f0c0 x1874671476784640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828085.962776:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476784640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828085.962809:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828085.962817:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828085.962825:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828085.962881:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828085.962928:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828085.962954:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828085/real 1787828085] req@ffff9a7890b1f480 x1874671476784768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828085.962981:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476784768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828085.962999:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828085.963006:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828085.963013:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828085.963074:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828085.963116:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828085.963138:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828085/real 1787828085] req@ffff9a7890b1f840 x1874671476784896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828085.963161:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476784896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828085.963181:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828085.963188:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828085.963196:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476784896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828085.963208:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828085.963248:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828085.963272:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828085/real 1787828085] req@ffff9a7890b1fc00 x1874671476785024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828085.963298:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476785024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828110 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828085.963318:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828085.963325:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828085.963332:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828086.986095:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828088.010052:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828089.034037:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828090.058034:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828091.081993:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828091.082041:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.082053:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828091.082059:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.082073:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2458 00000100:00080000:0.0:1787828091.082087:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828091.082096:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828091.082165:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476785152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828091.082209:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.082214:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828091.082216:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.082220:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2458 00000100:00080000:0.0:1787828091.082224:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828091.082227:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828091.082240:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828091.082258:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476785280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828091.082271:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.082275:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828091.082278:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.082288:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2458 00000100:00100000:3.0:1787828091.082291:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476785152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828091.082293:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828091.082303:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828091.082314:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828091.082328:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476785408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828091.082340:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.082344:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828091.082346:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.082351:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2458 00000100:00080000:0.0:1787828091.082354:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828091.082358:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828091.082365:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828091.082377:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476785536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828091.082385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828091.082389:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828091.082390:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828091.082394:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2458 00000100:00080000:0.0:1787828091.082399:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828091.082404:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828091.082415:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828091.082426:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828091.082473:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476785664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828091.082506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff9a7890a3fc00 x1874671476785152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828091.082534:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476785152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828091.082548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828091.082553:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828091.082563:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476785152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828091.082594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828091.082659:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828091.082669:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff9a7890a3c000 x1874671476785280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828091.082678:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476785280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828091.082684:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828091.082686:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828091.082705:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828091.082713:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828091.082733:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828091.082744:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff9a7890a3c3c0 x1874671476785408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828091.082757:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476785408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828091.082765:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828091.082768:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828091.082771:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828091.082778:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828091.082794:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828091.082804:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff9a7890a3f480 x1874671476785536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828091.082815:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476785536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828091.082826:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828091.082829:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828091.082833:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828091.082839:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828091.082870:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828091.082876:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828091/real 1787828091] req@ffff9a7890a3de00 x1874671476785664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828091.082901:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476785664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828091.082908:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828091.082910:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828091.082912:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828092.106101:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828093.130151:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828094.154000:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828095.178096:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828096.202098:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828096.202111:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828096.202118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828096.202146:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828096.202152:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2463 00000100:00080000:1.0:1787828096.202168:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828096.202180:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828096.202237:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476785792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828096.202262:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828096.202284:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828096.202286:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828096.202291:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2463 00000100:00080000:1.0:1787828096.202295:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828096.202300:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828096.202312:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828096.202330:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476785920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828096.202345:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828096.202354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476785792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828096.202368:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828096.202396:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828096.202400:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2463 00000100:00080000:1.0:1787828096.202403:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828096.202406:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828096.202410:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828096.202421:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476786048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828096.202429:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828096.202432:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828096.202434:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828096.202436:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2463 00000100:00080000:1.0:1787828096.202448:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828096.202451:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828096.202456:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787828096.202457:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828096.202468:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476786176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828096.202499:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828096.202503:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828096.202506:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828096.202511:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2463 00000100:00080000:1.0:1787828096.202516:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828096.202521:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828096.202528:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828096.202551:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476786304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828096.202586:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff9a7890b1c000 x1874671476785792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828096.202635:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476785792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828096.202651:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828096.202657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828096.202663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476785792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828096.202710:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828096.202749:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828096.202773:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff9a7890b1fc00 x1874671476785920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828096.202797:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476785920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828096.202809:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828096.202813:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828096.202817:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476785920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828096.202824:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828096.202882:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828096.202896:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff9a7890b1f840 x1874671476786048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828096.202910:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476786048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828096.202919:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828096.202923:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828096.202926:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828096.202933:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828096.202950:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828096.202976:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff9a7890b1f480 x1874671476786176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828096.202984:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476786176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828096.202991:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828096.202993:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828096.202995:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828096.202999:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828096.203020:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828096.203049:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828096/real 1787828096] req@ffff9a7890b1f0c0 x1874671476786304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828096.203062:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476786304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828096.203072:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828096.203075:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828096.203078:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828097.226121:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828098.250141:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828099.274181:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828100.298097:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828101.321902:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828101.321910:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828101.321914:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828101.321923:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2468 00000100:00080000:1.0:1787828101.321932:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828101.321939:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828101.321965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828101.321972:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476786432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828101.321993:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828101.321996:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828101.321997:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828101.322000:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2468 00000100:00080000:1.0:1787828101.322003:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828101.322005:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828101.322013:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828101.322022:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476786560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828101.322031:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476786432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828101.322032:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828101.322040:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828101.322041:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828101.322045:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2468 00000100:00080000:1.0:1787828101.322047:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828101.322050:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828101.322054:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828101.322062:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476786688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828101.322068:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828101.322071:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828101.322073:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828101.322075:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2468 00000100:00080000:1.0:1787828101.322085:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828101.322088:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828101.322093:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828101.322093:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828101.322105:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476786816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828101.322112:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828101.322115:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828101.322116:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828101.322119:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2468 00000100:00080000:1.0:1787828101.322121:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828101.322124:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828101.322128:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828101.322144:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476786944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828101.322144:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff9a7890b1e940 x1874671476786432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828101.322175:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476786432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828101.322185:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828101.322188:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828101.322191:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476786432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828101.322214:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828101.322239:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828101.322253:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff9a7890b1da40 x1874671476786560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828101.322266:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476786560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828101.322272:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828101.322275:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828101.322277:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828101.322282:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828101.322297:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828101.322304:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff9a7890b1e580 x1874671476786688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828101.322312:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476786688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828101.322317:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828101.322319:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828101.322322:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828101.322326:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828101.322337:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828101.322342:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff9a7890b1de00 x1874671476786816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828101.322350:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476786816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828101.322356:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828101.322358:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828101.322360:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828101.322363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828101.322378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828101.322389:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828101/real 1787828101] req@ffff9a7890b1c3c0 x1874671476786944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828101.322402:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476786944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828101.322412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828101.322416:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828101.322420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476786944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828102.346054:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828103.370094:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828104.394114:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828105.418022:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828106.442003:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828106.442021:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828106.442028:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828106.442045:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2473 00010000:00010000:2.0:1787828106.442078:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828106.442082:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828106.442095:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828106.442188:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476787072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828106.442225:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828106.442231:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828106.442236:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828106.442244:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2473 00000100:00080000:0.0:1787828106.442251:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828106.442262:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828106.442282:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828106.442339:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476787200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828106.442339:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476787072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828106.442367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828106.442372:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828106.442376:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828106.442382:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2473 00000100:00080000:0.0:1787828106.442387:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828106.442410:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828106.442419:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828106.442440:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476787328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828106.442454:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828106.442459:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828106.442461:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828106.442465:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2473 00000100:00080000:0.0:1787828106.442471:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828106.442475:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828106.442483:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828106.442521:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828106.442565:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476787456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828106.442585:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828106.442592:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828106.442596:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828106.442607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff9a7890a3da40 x1874671476787072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828106.442611:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2473 00000100:00080000:0.0:1787828106.442625:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828106.442655:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828106.442666:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828106.442675:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476787072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828106.442694:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476787584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828106.442715:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828106.442730:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828106.442744:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476787072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828106.442818:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828106.442871:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828106.442927:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff9a7890a3de00 x1874671476787200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828106.442954:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476787200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828106.442971:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828106.442978:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828106.442985:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828106.442999:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828106.443062:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828106.443081:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff9a7890a3f480 x1874671476787328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828106.443097:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476787328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828106.443115:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828106.443138:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828106.443143:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828106.443151:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828106.443179:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828106.443192:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff9a7890a3c3c0 x1874671476787456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828106.443207:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476787456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828106.443224:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828106.443231:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828106.443237:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828106.443249:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828106.443287:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828106.443311:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828106/real 1787828106] req@ffff9a7890a3c000 x1874671476787584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828106.443348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476787584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828106.443360:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828106.443364:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828106.443369:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828107.466079:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828108.490002:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828109.514113:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828110.538071:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828111.561975:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828111.561992:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828111.561999:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828111.562016:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2479 00000100:00080000:0.0:1787828111.562033:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828111.562044:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828111.562118:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476787712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828111.562152:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828111.562160:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828111.562168:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828111.562172:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828111.562178:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2479 00000100:00080000:0.0:1787828111.562183:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828111.562188:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828111.562204:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828111.562223:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476787840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828111.562238:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828111.562247:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828111.562251:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828111.562251:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476787712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828111.562260:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2479 00000100:00080000:0.0:1787828111.562271:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828111.562277:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828111.562283:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828111.562298:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476787968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828111.562310:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828111.562315:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828111.562319:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828111.562323:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2479 00000100:00080000:0.0:1787828111.562328:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828111.562332:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828111.562339:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828111.562360:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476788096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828111.562372:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828111.562376:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828111.562379:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828111.562391:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2479 00000800:00000100:3.0:1787828111.562394:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828111.562397:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828111.562404:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828111.562410:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828111.562426:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476788224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828111.562508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828111/real 1787828111] req@ffff9a7890a3fc00 x1874671476787712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828111.562537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476787712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828111.562553:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828111.562560:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828111.562565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476787712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828111.562601:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828111.562633:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828111.562649:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828111/real 1787828111] req@ffff9a7890a3c000 x1874671476787840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828111.562665:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476787840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828111.562675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828111.562679:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828111.562684:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828111.562690:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828111.562709:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828111.562721:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828111/real 1787828111] req@ffff9a7890a3c3c0 x1874671476787968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828111.562736:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476787968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828111.562746:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828111.562749:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828111.562753:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476787968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828111.562759:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828111.562778:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828111.562789:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828111/real 1787828111] req@ffff9a7890a3f480 x1874671476788096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828111.562802:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476788096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828111.562814:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828111.562818:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828111.562821:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828111.562828:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828111.562888:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828111.562909:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828111/real 1787828111] req@ffff9a7890a3de00 x1874671476788224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828111.562933:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476788224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828111.562951:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828111.562958:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828111.562965:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828112.586046:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828113.610063:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828114.634084:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828115.658046:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828116.681955:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828116.681969:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828116.681976:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828116.681992:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2484 00000100:00080000:0.0:1787828116.682007:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828116.682017:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828116.682069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476788352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828116.682094:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828116.682121:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828116.682128:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828116.682131:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828116.682137:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2484 00000100:00080000:0.0:1787828116.682141:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828116.682148:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828116.682161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828116.682186:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476788352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828116.682187:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476788480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828116.682215:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828116.682222:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828116.682226:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828116.682235:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2484 00000100:00080000:0.0:1787828116.682244:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828116.682252:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828116.682262:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828116.682287:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476788608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828116.682310:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828116.682323:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787828116.682325:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828116.682330:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828116.682344:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2484 00000100:00080000:0.0:1787828116.682353:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828116.682360:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828116.682369:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828116.682392:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476788736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828116.682408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828116/real 1787828116] req@ffff9a7890a3da40 x1874671476788352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828116.682409:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828116.682451:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828116.682458:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828116.682475:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2484 00000100:00080000:0.0:1787828116.682484:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828116.682490:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476788352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828116.682498:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828116.682516:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828116.682522:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828116.682531:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828116.682537:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476788352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828116.682543:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476788864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828116.682579:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828116.682632:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828116.682660:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828116/real 1787828116] req@ffff9a7890a3de00 x1874671476788480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828116.682689:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476788480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828116.682704:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828116.682709:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828116.682713:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828116.682723:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828116.682753:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828116.682768:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828116/real 1787828116] req@ffff9a7890a3f480 x1874671476788608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828116.682783:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476788608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828116.682793:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828116.682797:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828116.682801:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828116.682808:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828116.682827:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828116.682890:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828116/real 1787828116] req@ffff9a7890a3c3c0 x1874671476788736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828116.682915:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476788736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828116.682935:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828116.682942:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828116.682949:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828116.682978:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828116.683009:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828116.683025:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828116/real 1787828116] req@ffff9a7890a3c000 x1874671476788864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828116.683040:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476788864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828116.683053:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828116.683057:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828116.683061:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476788864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828117.706128:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828118.730085:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828119.754021:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828120.778119:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828121.802017:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828121.802030:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828121.802035:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828121.802049:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2489 00000100:00080000:2.0:1787828121.802063:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828121.802071:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828121.802083:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828121.802125:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476788992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828121.802161:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828121.802164:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828121.802166:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828121.802169:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2489 00000100:00080000:2.0:1787828121.802172:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828121.802175:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828121.802194:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828121.802207:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476789120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828121.802215:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476788992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828121.802217:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828121.802225:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828121.802227:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828121.802231:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2489 00000100:00080000:2.0:1787828121.802234:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828121.802237:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828121.802241:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828121.802251:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476789248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828121.802259:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828121.802263:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828121.802264:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828121.802267:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2489 00000100:00080000:2.0:1787828121.802290:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828121.802293:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828121.802297:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828121.802306:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476789376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828121.802314:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828121.802317:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828121.802318:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828121.802321:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2489 00000100:00080000:2.0:1787828121.802323:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828121.802326:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828121.802330:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828121.802341:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476789504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828121.802371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828121.802423:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828121.802479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828121.802506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828122.826175:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828123.850070:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828124.874212:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828124.874231:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787828124.874281:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828124.874302:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828124.874387:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828124.874411:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828124.874429:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787828124.874444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828121/real 1787828124] req@ffff9a78841d9a40 x1874671476789120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:0.0:1787828124.874446:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787828124.874491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476789120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828124.874511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828124.874520:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828124.874527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828124.874541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828121/real 1787828124] req@ffff9a78841db0c0 x1874671476789248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828124.874573:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476789248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828124.874586:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828124.874590:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828124.874596:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828124.874622:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828121/real 1787828124] req@ffff9a78841da1c0 x1874671476789376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828124.874640:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476789376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828124.874652:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828124.874658:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828124.874663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828124.874672:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828121/real 1787828124] req@ffff9a78841dbc00 x1874671476789504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828124.874690:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476789504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828124.874702:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828124.874706:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828124.874710:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789504:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828124.874718:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828121/real 1787828124] req@ffff9a78841d8780 x1874671476788992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828124.874734:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476788992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828124.874743:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828124.874747:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828124.874751:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476788992:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787828125.898126:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828126.921971:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828126.921985:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828126.921992:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828126.922006:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2494 00000100:00080000:0.0:1787828126.922020:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828126.922034:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828126.922069:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828126.922087:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476789632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828126.922126:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828126.922131:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828126.922133:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828126.922138:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2494 00000100:00080000:0.0:1787828126.922143:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828126.922148:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828126.922161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828126.922187:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476789632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828126.922188:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476789760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828126.922216:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828126.922220:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828126.922224:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828126.922232:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2494 00000100:00080000:0.0:1787828126.922241:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828126.922248:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828126.922258:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828126.922284:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476789888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828126.922298:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828126.922372:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828126.922386:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828126.922391:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828126.922408:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2494 00000100:00080000:0.0:1787828126.922417:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828126.922427:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828126.922445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828126/real 1787828126] req@ffff9a7890a3fc00 x1874671476789632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828126.922454:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828126.922494:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476790016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828126.922503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476789632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828126.922547:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828126.922554:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828126.922558:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828126.922567:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828126.922568:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787828126.922590:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2494 00000100:00100000:3.0:1787828126.922592:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476789632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828126.922609:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828126.922623:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828126.922633:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828126.922657:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828126.922681:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476790144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828126.922716:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828126.922752:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828126/real 1787828126] req@ffff9a7890a3c000 x1874671476789760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828126.922780:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476789760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828126.922808:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828126.922814:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828126.922820:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828126.923048:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828126.923096:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828126.923122:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828126/real 1787828126] req@ffff9a7890a3c3c0 x1874671476789888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828126.923147:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476789888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828126.923163:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828126.923169:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828126.923176:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476789888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828126.923188:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828126.923228:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828126.923252:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828126/real 1787828126] req@ffff9a7890a3f480 x1874671476790016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828126.923278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476790016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828126.923297:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828126.923303:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828126.923311:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828126.923344:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828126.923408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828126.923434:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828126/real 1787828126] req@ffff9a7890a3de00 x1874671476790144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828126.923458:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476790144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828126.923477:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828126.923483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828126.923490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828127.946116:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828128.970093:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828129.994130:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828131.018095:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828132.041997:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828132.042010:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828132.042016:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828132.042028:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2499 00000100:00080000:0.0:1787828132.042042:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828132.042053:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828132.042090:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828132.042120:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476790272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828132.042166:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828132.042170:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828132.042172:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828132.042176:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2499 00000100:00080000:0.0:1787828132.042180:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828132.042185:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828132.042197:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828132.042231:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476790272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828132.042233:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476790400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828132.042253:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828132.042256:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828132.042258:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828132.042264:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2499 00000100:00080000:0.0:1787828132.042267:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828132.042271:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828132.042277:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828132.042290:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476790528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828132.042300:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828132.042304:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828132.042305:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828132.042309:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2499 00000100:00080000:0.0:1787828132.042336:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828132.042345:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828132.042374:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787828132.042376:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828132.042387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476790656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828132.042394:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828132.042397:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828132.042398:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828132.042401:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2499 00000100:00080000:0.0:1787828132.042404:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828132.042406:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828132.042410:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828132.042419:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476790784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828132.042439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9a7890a3da40 x1874671476790272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828132.042456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476790272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828132.042465:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828132.042468:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828132.042472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476790272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828132.042511:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828132.042530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828132.042539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9a7890a3de00 x1874671476790400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828132.042547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476790400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828132.042553:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828132.042555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828132.042558:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828132.042561:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828132.042571:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828132.042577:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9a7890a3f480 x1874671476790528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828132.042584:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476790528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828132.042589:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828132.042591:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828132.042593:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828132.042596:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828132.042606:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828132.042611:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9a7890a3c3c0 x1874671476790656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828132.042618:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476790656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828132.042626:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828132.042629:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828132.042633:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828132.042639:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828132.042658:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828132.042670:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828132/real 1787828132] req@ffff9a7890a3c000 x1874671476790784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828132.042683:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476790784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828132.042691:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828132.042693:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828132.042695:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476790784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828133.066044:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828134.090006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828135.114137:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828136.138236:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828137.162001:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828137.162015:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828137.162020:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828137.162043:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2504 00000100:00080000:2.0:1787828137.162057:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828137.162067:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828137.162097:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828137.162119:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476790912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828137.162168:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828137.162171:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828137.162173:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828137.162177:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2504 00000100:00080000:2.0:1787828137.162181:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828137.162186:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828137.162195:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828137.162230:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476790912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828137.162231:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476791040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828137.162259:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828137.162263:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828137.162265:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828137.162272:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2504 00000100:00080000:2.0:1787828137.162295:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828137.162314:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828137.162322:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828137.162353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476791168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828137.162363:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828137.162379:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787828137.162380:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828137.162384:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828137.162391:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2504 00000100:00080000:2.0:1787828137.162396:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828137.162400:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828137.162405:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828137.162421:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476791296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828137.162435:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828137.162441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff9a78841d9e00 x1874671476790912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828137.162443:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828137.162462:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828137.162475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476790912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828137.162477:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2504 00000100:00080000:2.0:1787828137.162489:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828137.162493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828137.162499:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828137.162500:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:2.0:1787828137.162509:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828137.162509:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476790912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828137.162531:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476791424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828137.162562:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828137.162587:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828137.162599:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff9a78841db480 x1874671476791040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828137.162609:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476791040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828137.162617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828137.162620:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828137.162623:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828137.162627:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828137.162640:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828137.162647:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff9a78841d8000 x1874671476791168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828137.162657:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476791168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828137.162664:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828137.162666:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828137.162668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828137.162673:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828137.162685:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828137.162693:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff9a78841d83c0 x1874671476791296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828137.162703:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476791296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828137.162711:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828137.162713:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828137.162715:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828137.162720:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828137.162732:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828137.162739:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828137/real 1787828137] req@ffff9a78841da580 x1874671476791424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828137.162748:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476791424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828137.162756:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828137.162758:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828137.162760:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828138.186055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828139.210074:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828140.234094:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828141.257996:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828142.281947:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828142.281957:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828142.281962:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828142.281974:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2509 00010000:00010000:2.0:1787828142.281979:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828142.281985:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828142.281994:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828142.282030:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476791552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828142.282052:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828142.282057:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828142.282059:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828142.282063:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2509 00000100:00080000:0.0:1787828142.282067:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828142.282071:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828142.282081:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828142.282092:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476791552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828142.282094:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476791680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828142.282104:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828142.282107:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828142.282109:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828142.282113:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2509 00000100:00080000:0.0:1787828142.282116:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828142.282119:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828142.282123:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828142.282132:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476791808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828142.282164:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828142.282169:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787828142.282170:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828142.282171:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828142.282175:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2509 00000100:00080000:0.0:1787828142.282180:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828142.282185:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828142.282190:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828142.282202:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476791936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828142.282213:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828142.282217:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828142.282220:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828142.282220:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828142] req@ffff9a7890a3fc00 x1874671476791552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828142.282225:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2509 00000100:00080000:0.0:1787828142.282229:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828142.282233:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828142.282239:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828142.282261:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476791552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828142.282271:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476792064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828142.282277:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828142.282298:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828142.282302:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476791552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828142.282326:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828142.282347:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828142.282357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828142] req@ffff9a7890a3c000 x1874671476791680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828142.282366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476791680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828142.282371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828142.282373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828142.282375:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828142.282379:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828142.282390:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828142.282396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828142] req@ffff9a7890a3c3c0 x1874671476791808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828142.282403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476791808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828142.282408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828142.282410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828142.282412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828142.282416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828142.282426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828142.282431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828142] req@ffff9a7890a3f480 x1874671476791936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828142.282438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476791936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828142.282444:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828142.282446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828142.282447:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476791936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828142.282451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828142.282460:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828142.282465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828142/real 1787828142] req@ffff9a7890a3de00 x1874671476792064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828142.282472:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476792064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828142.282478:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828142.282480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828142.282481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828143.306036:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828144.330032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828145.353969:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828146.378037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828147.402017:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828147.402032:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828147.402039:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828147.402055:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2514 00000100:00080000:1.0:1787828147.402072:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828147.402101:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828147.402105:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828147.402151:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476792192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828147.402201:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828147.402229:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828147.402233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828147.402242:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2514 00000100:00080000:1.0:1787828147.402249:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828147.402266:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828147.402280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828147.402309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476792192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828147.402310:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476792320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828147.402353:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828147.402358:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828147.402361:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828147.402370:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2514 00000100:00080000:1.0:1787828147.402374:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828147.402379:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828147.402385:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828147.402401:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476792448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828147.402414:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828147.402419:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828147.402422:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828147.402429:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2514 00000100:00080000:1.0:1787828147.402439:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828147.402478:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828147.402481:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787828147.402490:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828147.402520:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476792576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828147.402535:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828147.402539:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828147.402541:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828147.402564:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2514 00000100:00080000:1.0:1787828147.402573:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828147.402580:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828147.402591:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828147.402618:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff9a7890b1d680 x1874671476792192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828147.402621:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476792704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828147.402699:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476792192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828147.402716:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828147.402724:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828147.402732:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476792192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828147.402780:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828147.402808:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828147.402823:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff9a7890b1c3c0 x1874671476792320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828147.402836:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476792320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828147.402874:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828147.402879:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828147.402883:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828147.402890:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828147.402911:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828147.402936:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff9a7890b1de00 x1874671476792448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828147.402956:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476792448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828147.402980:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828147.402986:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828147.402992:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828147.403003:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828147.403034:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828147.403048:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff9a7890b1e580 x1874671476792576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828147.403060:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476792576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828147.403071:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828147.403075:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828147.403078:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828147.403085:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828147.403113:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828147.403123:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828147/real 1787828147] req@ffff9a7890b1da40 x1874671476792704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828147.403134:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476792704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828147.403144:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828147.403147:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828147.403152:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828148.426161:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828149.450153:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828150.474079:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828151.497977:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828152.521924:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828152.521935:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828152.521940:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828152.521950:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2520 00000100:00080000:3.0:1787828152.521961:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828152.521969:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828152.522041:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476792832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828152.522078:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828152.522083:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828152.522085:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828152.522091:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2520 00000100:00080000:3.0:1787828152.522095:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828152.522101:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828152.522113:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828152.522130:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476792960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828152.522146:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476792832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828152.522149:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828152.522164:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828152.522168:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828152.522199:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2520 00000100:00080000:3.0:1787828152.522206:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828152.522215:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828152.522227:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828152.522255:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476793088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828152.522263:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828152.522287:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828152.522293:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828152.522296:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828152.522304:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2520 00000100:00080000:3.0:1787828152.522310:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828152.522315:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828152.522325:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828152.522343:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828152/real 1787828152] req@ffff9a78a6775680 x1874671476792832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828152.522348:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476793216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828152.522380:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828152.522386:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828152.522391:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828152.522404:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2520 00000100:00080000:3.0:1787828152.522413:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828152.522423:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828152.522430:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828152.522440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476792832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828152.522448:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476793344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828152.522483:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828152.522502:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828152.522511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476792832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828152.522546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828152.522575:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828152.522595:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828152/real 1787828152] req@ffff9a78a6776d00 x1874671476792960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828152.522614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476792960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828152.522626:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828152.522632:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828152.522638:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476792960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828152.522648:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793088:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787828152.522652:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:2.0:1787828152.522673:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828152.522688:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828152/real 1787828152] req@ffff9a78a67761c0 x1874671476793088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828152.522702:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476793088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828152.522720:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828152.522724:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828152.522728:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828152.522734:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828152.522752:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828152.522763:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828152/real 1787828152] req@ffff9a78a6774780 x1874671476793216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828152.522776:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476793216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828152.522788:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828152.522791:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828152.522795:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828152.522802:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828152.522819:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828152.522872:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828152/real 1787828152] req@ffff9a78a67770c0 x1874671476793344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828152.522889:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476793344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828152.522901:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828152.522905:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828152.522910:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828153.546020:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828154.570020:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828155.594083:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828156.618036:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828157.641993:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828157.642011:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828157.642020:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828157.642050:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2525 00010000:00010000:2.0:1787828157.642060:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828157.642068:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828157.642083:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828157.642189:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476793472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828157.642216:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828157.642220:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828157.642221:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828157.642243:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2525 00000100:00080000:1.0:1787828157.642249:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828157.642256:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828157.642269:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828157.642296:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476793472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828157.642305:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476793600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828157.642323:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828157.642328:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828157.642330:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828157.642336:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2525 00000100:00080000:1.0:1787828157.642340:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828157.642344:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828157.642349:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828157.642367:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476793728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828157.642378:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828157.642382:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828157.642384:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828157.642390:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2525 00000100:00080000:1.0:1787828157.642395:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828157.642399:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828157.642406:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787828157.642410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828157.642425:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476793856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828157.642441:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828157.642445:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828157.642448:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828157.642453:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2525 00000100:00080000:1.0:1787828157.642474:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828157.642478:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828157.642482:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828157.642489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828157/real 1787828157] req@ffff9a7890b1e940 x1874671476793472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828157.642492:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476793984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828157.642531:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476793472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828157.642543:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828157.642548:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828157.642552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476793472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828157.642577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828157.642600:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828157.642620:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828157/real 1787828157] req@ffff9a7890b1f0c0 x1874671476793600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828157.642629:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476793600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828157.642635:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828157.642637:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828157.642639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828157.642646:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828157.642661:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828157.642683:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828157/real 1787828157] req@ffff9a7890b1f480 x1874671476793728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828157.642695:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476793728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828157.642703:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828157.642706:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828157.642710:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828157.642716:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828157.642736:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828157.642747:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828157/real 1787828157] req@ffff9a7890b1f840 x1874671476793856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828157.642759:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476793856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828157.642769:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828157.642773:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828157.642776:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828157.642782:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828157.642809:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828157.642820:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828157/real 1787828157] req@ffff9a7890b1fc00 x1874671476793984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828157.642829:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476793984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828157.642836:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828157.642838:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828157.642840:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476793984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828158.666012:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828159.690013:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828160.714239:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828161.738129:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828162.761979:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828162.762010:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828162.762019:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828162.762023:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828162.762031:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2530 00000100:00080000:2.0:1787828162.762039:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828162.762055:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828162.762086:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476794112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828162.762140:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828162.762143:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828162.762144:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828162.762147:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2530 00000100:00080000:2.0:1787828162.762150:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828162.762154:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828162.762162:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828162.762174:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476794112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828162.762175:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476794240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828162.762194:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828162.762198:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828162.762200:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828162.762202:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2530 00000100:00080000:2.0:1787828162.762204:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828162.762207:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828162.762210:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828162.762217:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476794368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828162.762225:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828162.762227:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828162.762229:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828162.762231:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2530 00000100:00080000:2.0:1787828162.762240:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828162.762252:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828162.762256:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828162.762259:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828162.762267:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476794496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828162.762275:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828162.762278:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828162.762279:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828162.762283:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2530 00000100:00080000:2.0:1787828162.762285:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828162.762287:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828162.762291:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828162.762305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476794624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828162.762305:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828162/real 1787828162] req@ffff9a78841d8b40 x1874671476794112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828162.762344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476794112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828162.762353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828162.762356:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828162.762359:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476794112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828162.762388:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828162.762403:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828162.762411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828162/real 1787828162] req@ffff9a78841db840 x1874671476794240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828162.762420:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476794240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828162.762425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828162.762428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828162.762430:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828162.762433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828162.762450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828162.762457:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828162/real 1787828162] req@ffff9a78841d92c0 x1874671476794368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828162.762469:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476794368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828162.762474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828162.762476:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828162.762478:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828162.762482:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828162.762494:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828162.762500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828162/real 1787828162] req@ffff9a78841d9680 x1874671476794496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828162.762512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476794496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828162.762521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828162.762525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828162.762528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828162.762535:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828162.762555:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828162.762565:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828162/real 1787828162] req@ffff9a78841da940 x1874671476794624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828162.762573:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476794624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828162.762584:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828162.762586:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828162.762588:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828163.785950:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828164.810146:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828165.834069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828166.858049:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828167.882011:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787828167.882015:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828167.882024:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828167.882029:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828167.882040:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2535 00000100:00080000:1.0:1787828167.882051:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828167.882058:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828167.882140:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476794752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828167.882178:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828167.882187:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828167.882189:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828167.882195:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2535 00000100:00080000:1.0:1787828167.882200:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828167.882205:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828167.882218:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828167.882237:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476794880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828167.882253:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828167.882258:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828167.882261:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828167.882270:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476794752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828167.882271:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2535 00000100:00080000:1.0:1787828167.882279:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828167.882308:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828167.882315:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828167.882328:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476795008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828167.882339:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828167.882343:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828167.882346:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828167.882350:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2535 00000100:00080000:1.0:1787828167.882354:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828167.882358:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828167.882364:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828167.882374:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476795136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828167.882403:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828167.882407:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828167.882410:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828167.882418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828167.882437:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2535 00000100:00080000:1.0:1787828167.882444:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828167.882451:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828167.882458:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828167.882476:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476795264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828167.882519:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828167/real 1787828167] req@ffff9a7890b1c000 x1874671476794752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828167.882556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476794752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828167.882576:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828167.882584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828167.882591:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476794752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828167.882642:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828167.882677:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828167.882697:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828167/real 1787828167] req@ffff9a7890b1fc00 x1874671476794880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828167.882716:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476794880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828167.882730:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828167.882735:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828167.882740:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476794880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828167.882750:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828167.882778:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828167.882795:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828167/real 1787828167] req@ffff9a7890b1f840 x1874671476795008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828167.882813:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476795008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828167.882825:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828167.882830:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828167.882835:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828167.882844:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828167.882872:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828167.882922:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828167/real 1787828167] req@ffff9a7890b1f480 x1874671476795136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828167.882941:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476795136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828167.882955:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828167.882960:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828167.882965:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828167.882973:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828167.883003:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828167.883020:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828167/real 1787828167] req@ffff9a7890b1f0c0 x1874671476795264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828167.883039:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476795264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828167.883053:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828167.883058:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828167.883080:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828168.906256:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828169.930088:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828170.954123:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828171.978055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828173.002058:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828173.002072:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828173.002078:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828173.002165:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2540 00000100:00080000:3.0:1787828173.002180:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828173.002190:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828173.002306:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476795392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828173.002345:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828173.002349:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828173.002351:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828173.002355:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2540 00000100:00080000:3.0:1787828173.002358:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828173.002362:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828173.002375:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828173.002388:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476795520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828173.002405:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828173.002408:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828173.002410:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828173.002435:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2540 00000100:00080000:3.0:1787828173.002438:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828173.002442:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828173.002446:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828173.002458:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476795648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828173.002472:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476795392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828173.002473:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828173.002490:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828173.002497:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828173.002504:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2540 00000100:00080000:3.0:1787828173.002507:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828173.002520:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828173.002524:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828173.002537:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476795776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828173.002546:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828173.002549:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828173.002553:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828173.002562:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2540 00000100:00080000:3.0:1787828173.002649:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828173.002656:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787828173.002664:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787828173.002665:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828173.002702:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476795904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828173.002761:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9a78a6776940 x1874671476795392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828173.002793:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476795392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828173.002807:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828173.002813:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828173.002819:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476795392:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787828173.002888:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828173.002934:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828173.002997:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828173.003061:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9a78a6776580 x1874671476795520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828173.003089:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476795520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828173.003101:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828173.003106:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828173.003110:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828173.003199:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828173.003226:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828173.003239:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9a78a6774b40 x1874671476795648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828173.003252:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476795648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828173.003280:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828173.003284:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828173.003289:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828173.003297:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828173.003321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828173.003332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9a78a67752c0 x1874671476795776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828173.003352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476795776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828173.003384:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828173.003391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828173.003398:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828173.003427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828173.003458:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828173.003476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828173/real 1787828173] req@ffff9a78a6775a40 x1874671476795904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828173.003509:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476795904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828173.003521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828173.003525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828173.003530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476795904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828174.026115:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828175.050132:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828176.074129:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828177.098148:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828178.121977:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828178.121988:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828178.122013:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828178.122024:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2545 00000100:00080000:3.0:1787828178.122053:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828178.122060:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828178.122124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476796032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828178.122152:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828178.122155:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828178.122157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828178.122160:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2545 00000100:00080000:3.0:1787828178.122164:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828178.122167:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828178.122175:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828178.122185:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476796160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828178.122193:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828178.122200:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828178.122202:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828178.122206:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2545 00000100:00080000:3.0:1787828178.122209:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828178.122212:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828178.122218:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828178.122228:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476796288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828178.122236:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828178.122239:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828178.122242:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828178.122246:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2545 00000100:00080000:3.0:1787828178.122249:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828178.122251:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828178.122255:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828178.122267:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476796416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828178.122278:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476796032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828178.122282:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828178.122305:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828178.122308:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828178.122312:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2545 00000100:00080000:3.0:1787828178.122315:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828178.122318:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828178.122323:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828178.122335:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476796544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828178.122423:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828178.122520:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff9a78a6774000 x1874671476796032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828178.122541:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828178.122551:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476796032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828178.122570:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828178.122577:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828178.122583:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476796032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828178.122594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828178.122621:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828178.122636:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff9a78a6777480 x1874671476796160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828178.122652:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476796160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828178.122662:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828178.122665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828178.122669:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828178.122677:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828178.122696:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828178.122707:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff9a78a6775a40 x1874671476796288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828178.122722:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476796288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828178.122732:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828178.122735:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828178.122739:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828178.122778:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828178.122800:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828178.122812:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff9a78a67752c0 x1874671476796416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828178.122893:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476796416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828178.122907:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828178.122911:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828178.122915:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828178.122923:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828178.122947:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828178.122960:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828178/real 1787828178] req@ffff9a78a6774b40 x1874671476796544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828178.122974:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476796544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828178.122986:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828178.122990:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828178.123025:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828179.146118:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828180.170104:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828181.193985:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828182.218147:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828183.242080:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.242098:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828183.242106:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828183.242125:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2550 00000100:00080000:2.0:1787828183.242145:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828183.242160:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828183.242187:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828183.242293:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476796672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828183.242329:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.242332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828183.242333:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828183.242337:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2550 00000100:00080000:2.0:1787828183.242340:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828183.242344:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828183.242356:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828183.242370:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476796800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828183.242380:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.242383:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828183.242385:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828183.242389:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2550 00000100:00080000:2.0:1787828183.242392:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828183.242401:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828183.242403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476796672:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828183.242406:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828183.242451:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476796928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828183.242466:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.242472:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828183.242474:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828183.242480:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2550 00000100:00080000:2.0:1787828183.242486:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828183.242492:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828183.242500:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828183.242514:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476797056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828183.242526:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828183.242531:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828183.242539:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828183.242541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828183.242563:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2550 00000100:00080000:2.0:1787828183.242575:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828183.242584:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828183.242598:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828183.242645:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9a78841da940 x1874671476796672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828183.242648:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476797184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828183.242724:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476796672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828183.242746:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828183.242756:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828183.242765:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476796672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828183.242809:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828183.242882:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828183.242901:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9a78841d9680 x1874671476796800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828183.242917:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476796800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828183.242928:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828183.242932:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828183.242937:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828183.242948:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828183.242998:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828183.243023:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9a78841d92c0 x1874671476796928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828183.243037:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476796928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828183.243048:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828183.243052:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828183.243056:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476796928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828183.243063:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828183.243092:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828183.243103:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9a78841db840 x1874671476797056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828183.243117:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476797056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828183.243129:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828183.243132:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828183.243136:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828183.243143:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828183.243161:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828183.243172:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828183/real 1787828183] req@ffff9a78841d8b40 x1874671476797184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828183.243187:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476797184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828183.243198:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828183.243201:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828183.243205:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828184.266123:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828185.290148:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828186.313987:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828187.337962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828188.362037:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787828188.362037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828188.362050:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828188.362056:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.362070:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2555 00000100:00080000:1.0:1787828188.362087:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828188.362095:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828188.362147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476797312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828188.362181:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828188.362187:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828188.362190:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.362196:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2555 00000100:00080000:1.0:1787828188.362200:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828188.362205:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828188.362217:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828188.362217:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476797312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828188.362236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476797440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828188.362250:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828188.362255:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828188.362257:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.362263:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2555 00000100:00080000:1.0:1787828188.362289:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828188.362293:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828188.362298:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828188.362325:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476797568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828188.362333:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828188.362336:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828188.362338:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.362342:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2555 00000100:00080000:1.0:1787828188.362345:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828188.362349:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828188.362355:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828188.362366:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476797696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828188.362390:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828188.362392:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828188.362393:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828188.362396:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2555 00000100:00080000:1.0:1787828188.362399:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828188.362402:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797440:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787828188.362404:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828188.362412:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828188.362423:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476797824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828188.362445:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828188.362468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828188.362499:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828189.386025:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828190.410128:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828191.433963:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828191.434077:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787828191.434109:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828191.434123:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828191.434160:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828191.434169:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828191.434178:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828191.434186:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787828191.434226:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828191] req@ffff9a7890b1c3c0 x1874671476797824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828191.434247:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476797824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828191.434257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828191.434261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828191.434264:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797824:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828191.434270:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828191] req@ffff9a7890b1da40 x1874671476797440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828191.434282:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476797440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828191.434291:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828191.434294:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828191.434298:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828191.434305:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828191] req@ffff9a7890b1e580 x1874671476797568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828191.434318:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476797568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828191.434326:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828191.434330:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828191.434333:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828191.434339:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828191] req@ffff9a7890b1de00 x1874671476797696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828191.434352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476797696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828191.434362:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828191.434366:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828191.434369:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476797696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828191.434376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828188/real 1787828191] req@ffff9a7890b1e940 x1874671476797312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828191.434389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476797312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828191.434398:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828191.434401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828191.434404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476797312:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787828192.457998:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828193.481916:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828193.481924:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828193.481928:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828193.481937:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2560 00000100:00080000:1.0:1787828193.481946:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828193.481953:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828193.481953:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828193.481990:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476797952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828193.482011:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828193.482014:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828193.482015:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828193.482018:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2560 00000100:00080000:1.0:1787828193.482021:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828193.482023:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828193.482031:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828193.482041:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476798080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828193.482049:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828193.482054:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828193.482054:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476797952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828193.482057:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828193.482065:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2560 00000100:00080000:1.0:1787828193.482067:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828193.482069:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828193.482072:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828193.482085:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476798208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828193.482095:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828193.482097:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828193.482099:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828193.482101:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2560 00000100:00080000:1.0:1787828193.482110:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828193.482113:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828193.482117:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787828193.482123:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828193.482129:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476798336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828193.482137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828193.482139:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828193.482141:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828193.482144:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2560 00000100:00080000:1.0:1787828193.482146:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828193.482148:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828193.482151:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828193.482159:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476798464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828193.482175:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828193/real 1787828193] req@ffff9a7890b1d680 x1874671476797952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828193.482195:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476797952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828193.482206:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828193.482210:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828193.482213:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476797952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828193.482238:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828193.482262:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828193.482277:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828193/real 1787828193] req@ffff9a7890b1e940 x1874671476798080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828193.482291:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476798080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828193.482299:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828193.482302:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828193.482308:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828193.482314:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828193.482348:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828193.482362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828193/real 1787828193] req@ffff9a7890b1de00 x1874671476798208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828193.482374:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476798208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828193.482383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828193.482386:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828193.482390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828193.482396:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828193.482417:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828193.482428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828193/real 1787828193] req@ffff9a7890b1e580 x1874671476798336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828193.482439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476798336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828193.482448:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828193.482451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828193.482454:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828193.482460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828193.482479:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828193.482490:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828193/real 1787828193] req@ffff9a7890b1da40 x1874671476798464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828193.482501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476798464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828193.482511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828193.482513:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828193.482516:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828194.506049:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828195.530076:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828196.553962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828197.577999:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828198.601946:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828198.601957:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828198.601962:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828198.601974:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2566 00000100:00080000:1.0:1787828198.601986:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828198.601994:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828198.602013:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828198.602039:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476798592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828198.602065:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828198.602068:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828198.602070:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828198.602073:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2566 00000100:00080000:1.0:1787828198.602076:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828198.602079:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828198.602089:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828198.602100:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476798592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828198.602103:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476798720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828198.602114:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828198.602117:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828198.602118:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828198.602121:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2566 00000100:00080000:1.0:1787828198.602123:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828198.602125:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828198.602128:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828198.602137:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476798848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828198.602144:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828198.602146:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828198.602148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828198.602150:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2566 00000100:00080000:1.0:1787828198.602166:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828198.602170:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828198.602175:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787828198.602179:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828198.602188:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476798976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828198.602197:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828198.602200:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828198.602201:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828198.602204:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2566 00000100:00080000:1.0:1787828198.602207:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828198.602209:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828198.602213:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828198.602220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476799104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828198.602242:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828198/real 1787828198] req@ffff9a7890b1c3c0 x1874671476798592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828198.602259:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476798592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828198.602268:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828198.602271:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828198.602274:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476798592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828198.602298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828198.602313:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828198.602322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828198/real 1787828198] req@ffff9a7890b1f0c0 x1874671476798720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828198.602330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476798720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828198.602335:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828198.602338:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828198.602340:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828198.602343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828198.602353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828198.602359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828198/real 1787828198] req@ffff9a7890b1f480 x1874671476798848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828198.602367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476798848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828198.602372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828198.602374:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828198.602376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828198.602379:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828198.602389:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828198.602396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828198/real 1787828198] req@ffff9a7890b1f840 x1874671476798976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828198.602409:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476798976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828198.602419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828198.602424:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828198.602427:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476798976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828198.602433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828198.602454:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828198.602471:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828198/real 1787828198] req@ffff9a7890b1fc00 x1874671476799104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828198.602479:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476799104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828198.602491:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828198.602494:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828198.602496:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828199.625984:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828200.649978:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828201.673963:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828202.697986:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828203.721952:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828203.721960:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828203.721965:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828203.721973:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2571 00000100:00080000:0.0:1787828203.721983:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828203.721989:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828203.722006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828203.722024:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476799232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828203.722050:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828203.722052:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828203.722054:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828203.722058:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2571 00000100:00080000:0.0:1787828203.722060:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828203.722063:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828203.722072:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828203.722077:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476799232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828203.722085:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476799360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828203.722094:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828203.722097:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828203.722098:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828203.722101:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2571 00000100:00080000:0.0:1787828203.722104:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828203.722107:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828203.722112:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828203.722123:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476799488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828203.722134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1787828203.722136:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828203.722141:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828203.722144:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828203.722151:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2571 00000100:00080000:0.0:1787828203.722156:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828203.722161:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828203.722166:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828203.722179:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828203/real 1787828203] req@ffff9a7890a3da40 x1874671476799232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828203.722180:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476799616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828203.722206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828203.722208:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:2.0:1787828203.722211:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476799232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828203.722212:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828203.722221:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2571 00000100:00080000:0.0:1787828203.722226:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828203.722232:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828203.722236:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787828203.722237:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828203.722245:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828203.722246:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476799744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828203.722255:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476799232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828203.722282:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828203.722300:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828203.722310:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828203/real 1787828203] req@ffff9a7890a3de00 x1874671476799360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828203.722319:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476799360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828203.722328:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828203.722331:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828203.722333:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828203.722337:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828203.722353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828203.722364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828203/real 1787828203] req@ffff9a7890a3f480 x1874671476799488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828203.722377:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476799488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828203.722386:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828203.722389:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828203.722393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828203.722398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828203.722413:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828203.722420:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828203/real 1787828203] req@ffff9a7890a3c3c0 x1874671476799616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828203.722428:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476799616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828203.722434:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828203.722436:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828203.722438:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828203.722442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828203.722451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828203.722457:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828203/real 1787828203] req@ffff9a7890a3c000 x1874671476799744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828203.722464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476799744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828203.722469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828203.722471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828203.722473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476799744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828204.746021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828205.769952:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828206.793950:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828207.818006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828208.841908:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828208.841917:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828208.841920:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828208.841928:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2576 00000100:00080000:3.0:1787828208.841938:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828208.841945:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828208.841976:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476799872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828208.842001:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828208.842004:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828208.842005:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828208.842008:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2576 00000100:00080000:3.0:1787828208.842010:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828208.842013:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828208.842021:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828208.842034:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476800000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828208.842035:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476799872:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828208.842054:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828208.842056:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828208.842058:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828208.842060:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2576 00000100:00080000:3.0:1787828208.842062:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828208.842065:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828208.842068:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828208.842076:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476800128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828208.842082:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828208.842085:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828208.842086:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828208.842088:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2576 00000100:00080000:3.0:1787828208.842090:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828208.842093:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828208.842097:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787828208.842100:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828208.842109:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476800256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828208.842116:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828208.842118:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828208.842119:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828208.842122:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2576 00000100:00080000:3.0:1787828208.842125:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828208.842128:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828208.842131:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828208.842141:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828208/real 1787828208] req@ffff9a78a6776580 x1874671476799872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828208.842142:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476800384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828208.842169:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476799872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828208.842182:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828208.842186:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828208.842192:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476799872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828208.842230:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800000:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787828208.842242:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:0.0:1787828208.842265:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828208.842279:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828208/real 1787828208] req@ffff9a78a6776940 x1874671476800000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828208.842291:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476800000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828208.842299:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828208.842302:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828208.842315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828208.842321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828208.842352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828208.842372:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828208/real 1787828208] req@ffff9a78a67770c0 x1874671476800128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828208.842397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476800128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828208.842423:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828208.842426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828208.842435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828208.842442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828208.842461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828208.842490:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828208/real 1787828208] req@ffff9a78a6774780 x1874671476800256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828208.842499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476800256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828208.842506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828208.842508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828208.842510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828208.842513:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828208.842527:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828208.842534:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828208/real 1787828208] req@ffff9a78a67761c0 x1874671476800384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828208.842541:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476800384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828208.842546:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828208.842548:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828208.842550:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828209.866000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828210.889941:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828211.914012:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828212.938008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828213.961921:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828213.961929:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828213.961933:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828213.961940:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2581 00000100:00080000:3.0:1787828213.961949:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828213.961955:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828213.961986:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476800512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828213.962009:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828213.962012:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828213.962013:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828213.962015:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2581 00000100:00080000:3.0:1787828213.962018:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828213.962020:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828213.962027:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828213.962036:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476800640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828213.962043:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828213.962047:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828213.962049:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828213.962052:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2581 00000100:00080000:3.0:1787828213.962056:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828213.962057:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476800512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828213.962061:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828213.962068:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828213.962078:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476800768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828213.962085:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828213.962087:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828213.962089:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828213.962091:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2581 00000100:00080000:3.0:1787828213.962108:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828213.962111:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828213.962115:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828213.962118:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828213.962127:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476800896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828213.962135:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828213.962137:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828213.962138:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828213.962141:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2581 00000100:00080000:3.0:1787828213.962143:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828213.962145:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828213.962150:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828213.962160:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476801024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828213.962163:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828213/real 1787828213] req@ffff9a78a6776d00 x1874671476800512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828213.962187:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476800512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828213.962199:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828213.962204:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828213.962210:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476800512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828213.962219:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828213.962246:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828213.962260:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828213/real 1787828213] req@ffff9a78a6775680 x1874671476800640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787828213.962265:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828213.962272:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476800640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828213.962279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828213.962281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828213.962284:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828213.962303:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828213.962315:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828213.962322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828213/real 1787828213] req@ffff9a78a6775e00 x1874671476800768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828213.962329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476800768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828213.962334:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828213.962336:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828213.962339:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828213.962342:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828213.962353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828213.962358:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828213/real 1787828213] req@ffff9a78a6777840 x1874671476800896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828213.962365:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476800896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828213.962371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828213.962373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828213.962375:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476800896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828213.962378:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828213.962388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828213.962393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828213/real 1787828213] req@ffff9a78a6774f00 x1874671476801024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828213.962400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476801024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828213.962406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828213.962408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828213.962409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828214.986031:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828216.009955:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828217.033987:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828218.057919:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828219.081936:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828219.081945:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828219.081950:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828219.081960:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2586 00010000:00010000:2.0:1787828219.081969:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828219.081971:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828219.081979:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828219.082010:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476801152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828219.082034:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828219.082037:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828219.082038:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828219.082041:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2586 00000100:00080000:0.0:1787828219.082044:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828219.082047:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828219.082055:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828219.082062:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476801152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828219.082068:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476801280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828219.082077:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828219.082080:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828219.082081:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828219.082085:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2586 00000100:00080000:0.0:1787828219.082088:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828219.082091:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828219.082094:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828219.082102:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476801408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828219.082114:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828219.082116:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828219.082118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828219.082123:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2586 00000800:00000100:2.0:1787828219.082123:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828219.082129:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828219.082132:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828219.082135:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828219.082143:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476801536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828219.082150:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828219.082152:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828219.082154:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828219.082159:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2586 00000100:00080000:0.0:1787828219.082163:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828219.082167:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828219.082173:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828219.082176:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9a7890a3fc00 x1874671476801152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828219.082190:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476801664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828219.082201:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476801152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828219.082212:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828219.082215:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828219.082219:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476801152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828219.082238:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828219.082256:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828219.082265:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9a7890a3c000 x1874671476801280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828219.082273:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476801280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828219.082279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828219.082281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828219.082283:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828219.082287:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828219.082297:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828219.082302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9a7890a3c3c0 x1874671476801408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828219.082310:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476801408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828219.082320:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828219.082323:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828219.082325:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828219.082329:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828219.082338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828219.082344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9a7890a3f480 x1874671476801536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828219.082351:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476801536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828219.082357:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828219.082359:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828219.082361:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828219.082364:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828219.082373:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828219.082379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828219/real 1787828219] req@ffff9a7890a3de00 x1874671476801664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828219.082386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476801664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828219.082392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828219.082393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828219.082396:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828220.105967:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828221.129957:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828222.153960:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828223.177936:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828224.201973:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828224.201985:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:0.0:1787828224.201991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828224.201991:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828224.202008:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2591 00000100:00080000:1.0:1787828224.202022:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828224.202029:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828224.202072:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476801792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828224.202102:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828224.202105:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828224.202108:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828224.202113:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2591 00000100:00080000:1.0:1787828224.202118:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828224.202124:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828224.202138:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828224.202139:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476801792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828224.202179:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476801920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828224.202204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828224.202209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828224.202211:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828224.202216:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2591 00000800:00000100:2.0:1787828224.202219:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828224.202230:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828224.202239:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828224.202247:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828224.202270:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476802048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828224.202280:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828224.202284:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828224.202287:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828224.202294:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2591 00000100:00100000:2.0:1787828224.202300:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9a7890b1c000 x1874671476801792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828224.202302:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828224.202310:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828224.202326:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828224.202340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476801792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828224.202341:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476802176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828224.202411:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828224.202414:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828224.202415:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828224.202422:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828224.202427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787828224.202433:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2591 00000100:00080000:1.0:1787828224.202439:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828224.202439:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476801792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828224.202449:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828224.202454:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828224.202465:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476802304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828224.202475:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828224.202500:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828224.202510:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9a7890b1fc00 x1874671476801920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828224.202519:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476801920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828224.202525:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828224.202528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828224.202530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476801920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828224.202536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828224.202547:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828224.202553:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9a7890b1f840 x1874671476802048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828224.202560:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476802048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828224.202565:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828224.202567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828224.202569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828224.202572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828224.202582:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828224.202588:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9a7890b1f480 x1874671476802176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828224.202595:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476802176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828224.202601:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828224.202603:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828224.202605:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828224.202615:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828224.202625:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828224.202631:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828224/real 1787828224] req@ffff9a7890b1f0c0 x1874671476802304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828224.202639:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476802304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828224.202644:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828224.202647:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828224.202648:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828225.225935:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828226.249964:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828227.273952:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828228.297966:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828229.321921:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828229.321935:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828229.321940:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828229.321954:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2596 00000100:00080000:3.0:1787828229.321967:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828229.321976:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828229.322026:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476802432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828229.322062:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828229.322067:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828229.322069:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828229.322074:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2596 00000100:00080000:3.0:1787828229.322078:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828229.322083:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828229.322096:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828229.322116:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476802432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828229.322116:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476802560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828229.322135:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828229.322139:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828229.322141:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828229.322146:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2596 00000100:00080000:3.0:1787828229.322151:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828229.322156:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828229.322162:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828229.322179:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476802688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828229.322191:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828229.322195:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828229.322199:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1787828229.322200:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828229.322209:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2596 00000100:00080000:3.0:1787828229.322213:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828229.322218:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828229.322224:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828229.322238:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476802816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828229.322248:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828229.322253:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828229.322255:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828229.322260:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2596 00000100:00080000:3.0:1787828229.322264:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828229.322265:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff9a78a67743c0 x1874671476802432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828229.322273:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828229.322284:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828229.322291:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476802432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828229.322301:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476802944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828229.322303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.322312:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828229.322318:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476802432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828229.322349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828229.322377:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828229.322394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff9a78a6777c00 x1874671476802560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828229.322412:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828229.322415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476802560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828229.322424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.322427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828229.322429:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828229.322433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828229.322448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828229.322455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff9a78a6774f00 x1874671476802688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828229.322463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476802688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828229.322469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.322471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828229.322473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828229.322485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828229.322497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828229.322504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff9a78a6777840 x1874671476802816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828229.322511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476802816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828229.322524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.322526:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828229.322528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828229.322532:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828229.322542:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828229.322547:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828229/real 1787828229] req@ffff9a78a6775e00 x1874671476802944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828229.322555:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476802944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828229.322562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828229.322563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828229.322565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476802944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828230.346025:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828231.369949:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828232.393940:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828233.417959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828234.441924:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828234.441935:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828234.441940:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828234.441951:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2601 00000100:00080000:1.0:1787828234.441962:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828234.441969:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828234.441994:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828234.442009:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476803072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828234.442041:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828234.442045:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828234.442046:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828234.442051:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2601 00000100:00080000:1.0:1787828234.442054:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828234.442058:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828234.442066:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828234.442068:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476803072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828234.442083:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476803200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828234.442091:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828234.442094:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828234.442096:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828234.442099:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2601 00000100:00080000:1.0:1787828234.442102:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828234.442104:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828234.442109:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828234.442120:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476803328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828234.442127:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1787828234.442128:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828234.442131:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828234.442134:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828234.442137:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2601 00000100:00080000:1.0:1787828234.442141:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828234.442145:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828234.442150:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828234.442163:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476803456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828234.442174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828234.442178:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828234.442180:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828234.442182:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9a7890b1c3c0 x1874671476803072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828234.442188:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2601 00000100:00080000:1.0:1787828234.442197:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828234.442205:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828234.442206:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476803072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787828234.442211:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787828234.442223:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787828234.442225:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476803584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828234.442231:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828234.442239:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476803072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828234.442262:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828234.442277:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828234.442286:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9a7890b1da40 x1874671476803200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828234.442294:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476803200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828234.442300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828234.442302:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828234.442304:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828234.442308:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828234.442317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828234.442322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9a7890b1e580 x1874671476803328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828234.442329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476803328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828234.442334:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828234.442336:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828234.442338:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828234.442341:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828234.442351:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828234.442356:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9a7890b1de00 x1874671476803456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828234.442363:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476803456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828234.442369:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828234.442371:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828234.442373:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828234.442376:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828234.442384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828234.442390:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828234/real 1787828234] req@ffff9a7890b1e940 x1874671476803584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828234.442397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476803584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828234.442403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828234.442404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828234.442406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828235.466094:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828236.490116:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828237.514092:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828238.538040:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828239.561949:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828239.561958:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828239.561962:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828239.561974:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2607 00000100:00080000:2.0:1787828239.561986:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828239.561994:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828239.562022:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828239.562046:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476803712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828239.562072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828239.562075:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828239.562077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828239.562080:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2607 00000100:00080000:2.0:1787828239.562083:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828239.562087:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828239.562096:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828239.562108:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476803712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828239.562109:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476803840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828239.562124:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828239.562126:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828239.562127:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828239.562131:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2607 00000100:00080000:2.0:1787828239.562132:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828239.562135:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828239.562139:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828239.562148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476803968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828239.562155:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828239.562158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828239.562160:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828239.562163:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2607 00000100:00080000:2.0:1787828239.562173:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828239.562176:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828239.562180:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828239.562186:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828239.562191:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476804096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828239.562211:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828239.562215:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828239.562233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828239.562238:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2607 00000100:00080000:2.0:1787828239.562243:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828239.562260:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828239.562267:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828239.562270:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828239/real 1787828239] req@ffff9a78841da580 x1874671476803712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828239.562283:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476804224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828239.562299:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476803712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828239.562323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828239.562329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828239.562334:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476803712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828239.562358:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828239.562386:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828239.562402:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828239/real 1787828239] req@ffff9a78841d83c0 x1874671476803840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828239.562418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476803840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828239.562428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828239.562432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828239.562436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828239.562472:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828239.562501:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828239.562518:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828239/real 1787828239] req@ffff9a78841d8000 x1874671476803968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828239.562538:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476803968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828239.562551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828239.562555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828239.562559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476803968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828239.562588:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828239.562642:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828239.562655:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828239/real 1787828239] req@ffff9a78841db480 x1874671476804096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828239.562667:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476804096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828239.562677:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828239.562680:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828239.562683:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828239.562689:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828239.562707:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828239.562718:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828239/real 1787828239] req@ffff9a78841d9e00 x1874671476804224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828239.562731:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476804224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828239.562747:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828239.562751:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828239.562755:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828240.586021:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828241.609956:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828242.634127:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828243.658171:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828244.681986:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828244.681998:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828244.682004:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828244.682017:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2612 00000100:00080000:2.0:1787828244.682030:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828244.682041:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828244.682070:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828244.682101:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476804352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828244.682150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828244.682155:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828244.682158:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828244.682163:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2612 00000100:00080000:2.0:1787828244.682188:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828244.682195:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828244.682209:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828244.682228:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476804352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828244.682234:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476804480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828244.682260:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828244.682266:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828244.682270:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828244.682276:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2612 00000100:00080000:2.0:1787828244.682281:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828244.682285:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828244.682292:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828244.682323:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476804608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828244.682336:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828244.682339:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828244.682343:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828244.682352:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2612 00000800:00000100:0.0:1787828244.682353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828244.682361:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828244.682370:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828244.682395:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828244.682411:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476804736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828244.682425:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828244.682430:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828244.682433:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828244.682439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828244/real 1787828244] req@ffff9a78841d8780 x1874671476804352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828244.682444:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2612 00000100:00080000:2.0:1787828244.682456:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828244.682472:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828244.682482:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828244.682492:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476804352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828244.682498:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476804864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828244.682550:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828244.682571:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828244.682588:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476804352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828244.682624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828244.682649:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828244.682659:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828244/real 1787828244] req@ffff9a78841dbc00 x1874671476804480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828244.682669:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476804480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828244.682675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828244.682678:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828244.682681:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828244.682686:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828244.682698:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828244.682704:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828244/real 1787828244] req@ffff9a78841da1c0 x1874671476804608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828244.682711:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476804608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828244.682717:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828244.682719:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828244.682721:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828244.682725:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828244.682736:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828244.682742:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828244/real 1787828244] req@ffff9a78841db0c0 x1874671476804736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828244.682750:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476804736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828244.682757:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828244.682759:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828244.682760:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828244.682765:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828244.682775:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828244.682781:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828244/real 1787828244] req@ffff9a78841d9a40 x1874671476804864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828244.682788:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476804864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828244.682803:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828244.682805:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828244.682807:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476804864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828245.706097:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828246.730086:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828247.754063:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828248.778119:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828249.801988:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828249.801999:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828249.802004:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828249.802016:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2617 00000100:00080000:2.0:1787828249.802028:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828249.802036:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828249.802069:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828249.802081:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476804992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828249.802113:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828249.802118:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828249.802120:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828249.802124:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2617 00000100:00080000:2.0:1787828249.802127:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828249.802131:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828249.802142:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828249.802156:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476805120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828249.802168:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828249.802172:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828249.802174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828249.802178:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2617 00000100:00080000:2.0:1787828249.802183:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828249.802186:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476804992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828249.802191:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828249.802218:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828249.802234:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476805248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828249.802248:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828249.802252:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828249.802255:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828249.802260:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2617 00000100:00080000:2.0:1787828249.802278:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828249.802284:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828249.802290:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828249.802306:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828249.802309:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476805376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828249.802325:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828249.802330:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828249.802333:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828249.802339:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2617 00000100:00080000:2.0:1787828249.802342:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828249.802347:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828249.802353:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828249.802369:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476805504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828249.802393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828249/real 1787828249] req@ffff9a78841d9a40 x1874671476804992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828249.802433:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476804992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828249.802456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828249.802463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828249.802468:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476804992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828249.802480:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828249.802550:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828249.802570:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828249/real 1787828249] req@ffff9a78841db0c0 x1874671476805120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828249.802596:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476805120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828249.802608:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828249.802612:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828249.802616:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828249.802647:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828249.802671:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828249.802683:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828249/real 1787828249] req@ffff9a78841da1c0 x1874671476805248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828249.802699:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476805248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828249.802716:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828249.802722:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828249.802744:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828249.802755:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828249.802789:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828249.802821:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828249/real 1787828249] req@ffff9a78841dbc00 x1874671476805376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828249.802880:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476805376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828249.802909:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828249.802915:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828249.802951:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828249.802962:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828249.802995:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828249.803015:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828249/real 1787828249] req@ffff9a78841d8780 x1874671476805504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828249.803044:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476805504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828249.803055:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828249.803058:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828249.803062:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828250.826112:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828251.849948:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828252.874046:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828253.898102:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828254.922042:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828254.922056:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828254.922063:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828254.922077:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2622 00000100:00080000:1.0:1787828254.922091:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828254.922100:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828254.922171:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476805632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828254.922189:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828254.922230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828254.922237:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828254.922240:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828254.922245:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2622 00000100:00080000:1.0:1787828254.922250:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828254.922255:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828254.922268:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828254.922289:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476805760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828254.922314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828254.922319:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828254.922321:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828254.922326:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2622 00000100:00080000:1.0:1787828254.922330:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828254.922338:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828254.922343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476805632:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787828254.922346:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828254.922364:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476805888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828254.922386:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828254.922391:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828254.922394:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828254.922398:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2622 00000100:00080000:1.0:1787828254.922424:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828254.922429:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828254.922435:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828254.922450:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476806016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828254.922486:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828254.922492:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828254.922495:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828254.922503:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2622 00000100:00080000:1.0:1787828254.922509:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828254.922515:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828254.922523:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828254.922542:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828254.922550:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476806144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828254.922615:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828254.922677:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828254.922723:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828255.946081:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828256.970062:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828257.994226:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00010000:00010000:0.0:1787828257.994294:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828257.994336:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828257.994357:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828257.994423:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828257.994453:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828257.994465:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828257.994475:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787828257.994567:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828254/real 1787828257] req@ffff9a7890b1da40 x1874671476806144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828257.994609:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476806144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828257.994628:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828257.994635:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828257.994640:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806144:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828257.994650:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828254/real 1787828257] req@ffff9a7890b1e940 x1874671476805760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828257.994667:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476805760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828257.994678:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828257.994683:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828257.994688:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828257.994696:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828254/real 1787828257] req@ffff9a7890b1de00 x1874671476805888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828257.994713:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476805888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828257.994724:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828257.994729:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828257.994733:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476805888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828257.994741:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828254/real 1787828257] req@ffff9a7890b1e580 x1874671476806016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828257.994758:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476806016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828257.994770:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828257.994774:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828257.994778:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828257.994786:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828254/real 1787828257] req@ffff9a7890b1d680 x1874671476805632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828257.994804:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476805632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828257.994814:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828257.994819:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828257.994823:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476805632:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787828259.018049:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828260.041991:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828260.042007:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828260.042012:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828260.042027:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2627 00000100:00080000:1.0:1787828260.042041:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828260.042052:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828260.042097:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476806272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828260.042165:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828260.042170:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828260.042173:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828260.042180:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2627 00000100:00080000:1.0:1787828260.042185:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828260.042190:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828260.042203:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787828260.042221:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828260.042226:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476806400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828260.042260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828260.042264:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476806272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828260.042273:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828260.042280:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828260.042291:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2627 00000100:00080000:1.0:1787828260.042299:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828260.042307:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828260.042317:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828260.042343:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476806528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828260.042371:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828260.042385:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828260.042396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828260.042399:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828260.042405:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2627 00000100:00080000:1.0:1787828260.042411:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828260.042417:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828260.042438:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828260.042459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9a7890b1c3c0 x1874671476806272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828260.042472:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476806656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828260.042481:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828260.042486:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476806272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828260.042486:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828260.042490:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828260.042500:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2627 00000100:00080000:1.0:1787828260.042507:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828260.042515:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787828260.042518:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787828260.042521:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787828260.042526:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828260.042532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476806272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828260.042539:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476806784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828260.042561:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828260.042590:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828260.042605:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9a7890b1f0c0 x1874671476806400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828260.042619:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476806400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828260.042627:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828260.042631:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828260.042634:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828260.042640:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828260.042659:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828260.042670:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9a7890b1f480 x1874671476806528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828260.042682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476806528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828260.042690:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828260.042693:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828260.042697:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828260.042704:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828260.042725:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828260.042737:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9a7890b1f840 x1874671476806656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828260.042750:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476806656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828260.042760:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828260.042763:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828260.042767:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828260.042792:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828260.042813:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828260.042827:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9a7890b1fc00 x1874671476806784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828260.042870:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476806784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828260.042880:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828260.042883:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828260.042886:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476806784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828261.066082:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828262.089949:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828263.114025:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828264.138025:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828265.161963:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828265.161976:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828265.161981:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828265.161994:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2632 00000100:00080000:3.0:1787828265.162007:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828265.162016:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828265.162073:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476806912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828265.162108:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828265.162126:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828265.162128:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828265.162133:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2632 00000100:00080000:3.0:1787828265.162138:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828265.162154:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828265.162167:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828265.162184:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476807040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828265.162200:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828265.162217:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476806912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828265.162218:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828265.162225:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828265.162239:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2632 00000100:00080000:3.0:1787828265.162242:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828265.162247:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828265.162255:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828265.162273:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476807168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828265.162285:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828265.162290:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828265.162292:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828265.162298:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2632 00000100:00080000:3.0:1787828265.162303:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828265.162307:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828265.162316:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787828265.162317:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828265.162356:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476807296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828265.162368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828265.162372:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828265.162374:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828265.162380:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2632 00000100:00080000:3.0:1787828265.162426:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828265.162434:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828265.162447:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9a78a6775680 x1874671476806912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1787828265.162451:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828265.162479:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476807424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828265.162489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476806912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.162523:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.162530:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828265.162536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476806912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828265.162578:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828265.162628:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:1.0:1787828265.162633:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828265.162654:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9a78a6776d00 x1874671476807040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828265.162682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476807040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.162700:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.162707:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828265.162712:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828265.162721:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828265.162753:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828265.162776:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9a78a67761c0 x1874671476807168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828265.162813:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476807168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.162883:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.162892:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828265.162899:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828265.162911:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828265.162943:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828265.162958:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9a78a6774780 x1874671476807296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828265.162974:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476807296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.162986:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.162990:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828265.162994:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828265.163001:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828265.163019:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828265.163030:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9a78a67770c0 x1874671476807424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828265.163052:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476807424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828265.163071:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828265.163075:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828265.163079:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828266.186001:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828267.210060:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828268.234132:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828269.258152:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828270.281977:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.281988:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828270.282004:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.282015:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2637 00000100:00080000:1.0:1787828270.282025:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828270.282032:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828270.282070:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828270.282085:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476807552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828270.282122:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.282126:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828270.282127:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.282131:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2637 00000100:00080000:1.0:1787828270.282135:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828270.282139:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828270.282148:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828270.282157:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476807552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828270.282164:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476807680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828270.282174:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.282177:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828270.282178:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.282182:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2637 00000100:00080000:1.0:1787828270.282196:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828270.282199:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828270.282204:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828270.282215:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476807808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828270.282241:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.282245:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828270.282247:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.282255:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2637 00000100:00080000:1.0:1787828270.282265:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787828270.282266:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828270.282274:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828270.282284:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828270.282313:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476807936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828270.282321:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.282324:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828270.282326:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.282329:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2637 00000100:00080000:1.0:1787828270.282332:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828270.282336:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828270.282340:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828270.282366:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476808064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828270.282377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff9a7890b1c000 x1874671476807552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828270.282410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476807552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828270.282424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828270.282430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828270.282435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476807552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828270.282460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828270.282509:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828270.282523:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff9a7890b1fc00 x1874671476807680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828270.282553:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476807680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828270.282564:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828270.282587:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828270.282590:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828270.282599:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828270.282618:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828270.282628:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff9a7890b1f840 x1874671476807808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828270.282638:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476807808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828270.282644:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828270.282647:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828270.282649:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828270.282654:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828270.282666:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828270.282673:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff9a7890b1f480 x1874671476807936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828270.282682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476807936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828270.282689:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828270.282692:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828270.282694:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476807936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828270.282698:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828270.282711:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828270.282718:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828270] req@ffff9a7890b1f0c0 x1874671476808064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828270.282727:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476808064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828270.282734:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828270.282737:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828270.282739:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828271.306110:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828272.330125:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828273.354122:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828274.378067:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828275.402005:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.402026:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828275.402036:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.402058:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2642 00000100:00080000:1.0:1787828275.402082:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.402098:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828275.402107:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828275.402176:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476808192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828275.402222:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.402228:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828275.402231:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.402237:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2642 00000100:00080000:1.0:1787828275.402242:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.402248:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828275.402263:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828275.402279:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476808320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828275.402293:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.402297:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828275.402300:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.402305:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2642 00000100:00080000:1.0:1787828275.402311:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.402345:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828275.402353:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828275.402373:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476808448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828275.402385:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476808192:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828275.402393:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.402401:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828275.402403:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.402410:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2642 00000100:00080000:1.0:1787828275.402415:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.402420:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828275.402427:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828275.402443:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476808576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828275.402455:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828275.402461:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828275.402463:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828275.402467:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2642 00000100:00080000:1.0:1787828275.402484:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828275.402489:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828275.402495:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:0.0:1787828275.402504:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828275.402512:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476808704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828275.402607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9a7890b1c3c0 x1874671476808192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828275.402648:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476808192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828275.402671:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828275.402678:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828275.402683:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476808192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828275.402695:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828275.402737:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828275.402763:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9a7890b1d680 x1874671476808320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828275.402805:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476808320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828275.402823:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828275.402871:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828275.402877:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828275.402916:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828275.402958:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828275.402984:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9a7890b1e580 x1874671476808448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828275.403011:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476808448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828275.403029:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828275.403035:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828275.403043:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828275.403052:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828275.403082:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828275.403098:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9a7890b1de00 x1874671476808576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828275.403112:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476808576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828275.403134:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828275.403138:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828275.403142:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828275.403149:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828275.403168:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828275.403179:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9a7890b1e940 x1874671476808704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828275.403192:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476808704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828275.403204:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828275.403208:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828275.403211:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828276.425980:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828277.450001:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828278.473953:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828279.498059:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828280.521926:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828280.521936:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828280.521941:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828280.521951:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2648 00000100:00080000:1.0:1787828280.521962:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828280.521969:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828280.522004:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476808832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828280.522014:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828280.522031:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828280.522036:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828280.522038:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828280.522042:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2648 00000100:00080000:1.0:1787828280.522045:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828280.522049:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828280.522059:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828280.522072:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476808960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828280.522085:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828280.522088:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476808832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828280.522090:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828280.522094:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828280.522102:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2648 00000100:00080000:1.0:1787828280.522105:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828280.522109:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828280.522114:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828280.522125:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476809088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828280.522135:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828280.522139:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828280.522141:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828280.522145:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2648 00000100:00080000:1.0:1787828280.522149:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828280.522152:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828280.522158:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787828280.522158:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828280.522172:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476809216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828280.522182:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828280.522185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828280.522187:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828280.522193:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2648 00000100:00080000:1.0:1787828280.522197:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828280.522201:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828280.522204:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9a7890b1da40 x1874671476808832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787828280.522207:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828280.522224:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476809344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828280.522227:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476808832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828280.522255:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828280.522259:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828280.522263:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476808832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828280.522302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828280.522320:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828280.522330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9a7890b1e940 x1874671476808960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828280.522345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476808960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828280.522351:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828280.522353:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828280.522355:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476808960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828280.522360:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828280.522371:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828280.522378:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9a7890b1de00 x1874671476809088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828280.522385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476809088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828280.522390:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828280.522392:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828280.522394:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828280.522398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828280.522408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828280.522416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9a7890b1e580 x1874671476809216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828280.522429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476809216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828280.522438:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828280.522442:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828280.522445:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828280.522452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828280.522471:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828280.522481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9a7890b1d680 x1874671476809344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828280.522488:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476809344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828280.522495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828280.522497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828280.522499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828281.545940:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828282.569991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828283.594028:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828284.617934:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828285.642000:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828285.642008:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.642014:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828285.642018:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.642026:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2653 00000100:00080000:2.0:1787828285.642043:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828285.642049:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828285.642078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476809472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828285.642099:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.642105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828285.642107:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.642110:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2653 00000100:00080000:2.0:1787828285.642113:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828285.642116:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828285.642125:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828285.642130:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476809472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828285.642136:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476809600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828285.642145:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.642148:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828285.642150:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.642153:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2653 00000100:00080000:2.0:1787828285.642174:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828285.642180:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828285.642188:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828285.642205:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476809728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828285.642220:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.642230:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828285.642233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.642239:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2653 00000100:00080000:2.0:1787828285.642267:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787828285.642286:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828285.642291:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828285.642297:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828285.642307:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476809856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828285.642313:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.642315:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828285.642317:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.642320:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2653 00000100:00080000:2.0:1787828285.642322:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828285.642325:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828285.642329:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828285.642338:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476809984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828285.642344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9a78841d9e00 x1874671476809472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828285.642365:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476809472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828285.642373:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828285.642376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828285.642379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476809472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828285.642397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828285.642415:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828285.642424:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9a78841db480 x1874671476809600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828285.642432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476809600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828285.642438:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828285.642441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828285.642443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828285.642446:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828285.642456:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828285.642462:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9a78841d8000 x1874671476809728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828285.642469:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476809728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828285.642478:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828285.642480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828285.642482:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828285.642486:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828285.642496:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828285.642501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9a78841d83c0 x1874671476809856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828285.642508:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476809856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828285.642515:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828285.642516:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828285.642518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828285.642522:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828285.642532:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828285.642537:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9a78841da580 x1874671476809984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828285.642545:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476809984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828285.642550:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828285.642552:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828285.642554:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476809984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828286.665982:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828287.689939:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828288.713938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828289.737957:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828290.761944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.761956:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828290.761961:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.761972:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2658 00000100:00080000:2.0:1787828290.761986:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.761996:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828290.762018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828290.762043:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476810112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828290.762064:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.762067:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828290.762069:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.762072:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2658 00000100:00080000:2.0:1787828290.762075:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.762078:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828290.762088:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828290.762096:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476810112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828290.762099:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476810240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828290.762109:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.762111:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828290.762112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.762115:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2658 00000100:00080000:2.0:1787828290.762117:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.762119:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828290.762123:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828290.762132:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476810368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828290.762145:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.762149:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828290.762151:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.762155:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2658 00000100:00080000:2.0:1787828290.762162:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.762166:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828290.762173:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828290.762173:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828290.762192:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476810496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828290.762203:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.762207:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828290.762219:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.762224:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2658 00000100:00080000:2.0:1787828290.762229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.762234:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828290.762239:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828290.762245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9a78841d8b40 x1874671476810112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828290.762253:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476810624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828290.762267:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476810112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828290.762277:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828290.762281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828290.762284:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476810112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828290.762309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828290.762327:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828290.762336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9a78841db840 x1874671476810240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828290.762345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476810240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828290.762351:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828290.762354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828290.762356:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828290.762359:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828290.762371:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828290.762377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9a78841d92c0 x1874671476810368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828290.762384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476810368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828290.762389:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828290.762391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828290.762393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828290.762397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828290.762408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828290.762413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9a78841d9680 x1874671476810496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828290.762421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476810496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828290.762428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828290.762430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828290.762431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828290.762435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828290.762444:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828290.762450:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9a78841da940 x1874671476810624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828290.762462:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476810624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828290.762469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828290.762478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828290.762481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828291.786047:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828292.810058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828293.834017:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828294.857987:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828295.881940:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828295.881951:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828295.881956:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828295.881968:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2663 00000100:00080000:1.0:1787828295.881981:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828295.881990:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828295.881997:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828295.882042:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476810752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828295.882067:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828295.882070:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828295.882072:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828295.882075:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2663 00000100:00080000:1.0:1787828295.882079:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828295.882083:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828295.882093:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828295.882103:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476810752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828295.882105:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476810880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828295.882117:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828295.882120:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828295.882121:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828295.882125:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2663 00000100:00080000:1.0:1787828295.882128:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828295.882130:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828295.882133:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828295.882141:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476811008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828295.882149:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828295.882152:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828295.882154:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828295.882156:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2663 00000100:00080000:1.0:1787828295.882168:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828295.882171:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828295.882186:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828295.882196:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828295.882198:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476811136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828295.882206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828295.882209:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828295.882211:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828295.882215:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2663 00000100:00080000:1.0:1787828295.882225:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828295.882228:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828295.882231:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828295.882240:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476811264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828295.882256:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9a7890b1c3c0 x1874671476810752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828295.882273:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476810752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828295.882282:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828295.882285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828295.882288:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476810752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828295.882311:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828295.882328:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828295.882336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9a7890b1f0c0 x1874671476810880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828295.882349:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476810880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828295.882356:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828295.882358:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828295.882360:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476810880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828295.882363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828295.882374:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828295.882380:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9a7890b1f480 x1874671476811008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828295.882387:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476811008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828295.882393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828295.882394:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828295.882396:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828295.882399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828295.882410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828295.882415:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9a7890b1f840 x1874671476811136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828295.882422:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476811136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828295.882434:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828295.882435:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828295.882437:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828295.882440:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828295.882449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828295.882455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9a7890b1fc00 x1874671476811264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828295.882461:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476811264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828295.882469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828295.882471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828295.882472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828296.905973:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828297.929973:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828298.953988:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828299.977967:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828301.001958:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828301.001968:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828301.001973:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828301.001984:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2668 00000100:00080000:3.0:1787828301.001996:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828301.001997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828301.002014:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828301.002052:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476811392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828301.002073:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828301.002077:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828301.002079:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828301.002083:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2668 00000100:00080000:3.0:1787828301.002088:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828301.002093:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828301.002102:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828301.002107:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476811392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828301.002114:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476811520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828301.002124:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828301.002126:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828301.002128:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828301.002132:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2668 00000100:00080000:3.0:1787828301.002135:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828301.002161:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828301.002166:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828301.002178:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476811648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828301.002188:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828301.002194:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828301.002196:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828301.002198:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828301.002202:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2668 00000100:00080000:3.0:1787828301.002205:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828301.002208:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828301.002212:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828301.002221:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476811776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828301.002230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828301.002233:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:0.0:1787828301.002235:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff9a78a6776940 x1874671476811392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828301.002243:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828301.002246:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2668 00000100:00080000:3.0:1787828301.002249:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828301.002253:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476811392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828301.002253:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828301.002258:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828301.002267:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476811904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828301.002270:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828301.002274:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828301.002278:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476811392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828301.002301:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828301.002323:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828301.002333:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff9a78a6776580 x1874671476811520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828301.002341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476811520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828301.002347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828301.002349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828301.002351:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828301.002356:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828301.002373:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828301.002385:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff9a78a6774b40 x1874671476811648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828301.002398:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476811648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828301.002406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828301.002410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828301.002414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828301.002419:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828301.002435:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828301.002443:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff9a78a67752c0 x1874671476811776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828301.002451:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476811776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828301.002457:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828301.002459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828301.002461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828301.002465:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828301.002474:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828301.002480:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828301/real 1787828301] req@ffff9a78a6775a40 x1874671476811904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828301.002487:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476811904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828301.002493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828301.002495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828301.002497:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476811904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828302.026036:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828303.049948:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828304.074012:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828305.097973:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828306.121969:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828306.121982:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828306.121988:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828306.122001:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2673 00000100:00080000:2.0:1787828306.122015:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828306.122021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828306.122024:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828306.122077:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476812032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828306.122120:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828306.122125:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828306.122128:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828306.122134:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2673 00000100:00080000:2.0:1787828306.122138:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828306.122144:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828306.122156:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828306.122159:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476812032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828306.122176:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476812160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828306.122191:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828306.122195:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828306.122197:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828306.122203:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2673 00000100:00080000:2.0:1787828306.122225:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828306.122232:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828306.122239:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828306.122245:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828306.122266:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476812288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828306.122280:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828306.122284:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828306.122287:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828306.122293:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2673 00000100:00080000:2.0:1787828306.122299:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828306.122300:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9a78841da940 x1874671476812032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828306.122307:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828306.122318:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828306.122337:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476812416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828306.122343:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476812032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828306.122372:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828306.122384:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828306.122386:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828306.122388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828306.122396:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2673 00000100:00080000:0.0:1787828306.122398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787828306.122404:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828306.122412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476812032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828306.122415:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828306.122427:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828306.122443:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476812544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828306.122451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828306.122496:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828306.122513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9a78841d9680 x1874671476812160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828306.122528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476812160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828306.122539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828306.122543:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828306.122547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828306.122554:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828306.122577:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828306.122590:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9a78841d92c0 x1874671476812288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828306.122605:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476812288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828306.122615:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828306.122618:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828306.122622:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828306.122629:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828306.122653:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828306.122665:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9a78841db840 x1874671476812416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828306.122688:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476812416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828306.122699:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828306.122703:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828306.122707:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828306.122723:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828306.122743:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828306.122753:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828306/real 1787828306] req@ffff9a78841d8b40 x1874671476812544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828306.122765:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476812544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828306.122782:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828306.122785:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828306.122788:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828307.146002:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828308.169974:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828309.194034:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828310.218069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828311.242007:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828311.242019:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828311.242026:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.242039:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2678 00000100:00080000:2.0:1787828311.242053:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.242063:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828311.242168:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476812672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828311.242202:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828311.242208:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828311.242208:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828311.242214:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.242221:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2678 00000100:00080000:2.0:1787828311.242241:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.242245:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828311.242259:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828311.242272:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476812800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828311.242280:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828311.242286:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787828311.242287:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476812672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828311.242289:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.242298:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2678 00000100:00080000:2.0:1787828311.242301:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.242304:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828311.242307:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828311.242319:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476812928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828311.242331:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828311.242335:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828311.242336:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.242339:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2678 00000100:00080000:2.0:1787828311.242357:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.242360:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828311.242363:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828311.242370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828311.242375:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476813056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828311.242383:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828311.242387:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828311.242388:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828311.242392:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2678 00000100:00080000:2.0:1787828311.242400:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828311.242402:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828311.242405:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828311.242417:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476813184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828311.242444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9a78841da580 x1874671476812672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.242462:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476812672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.242472:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.242476:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828311.242479:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476812672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828311.242504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828311.242522:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828311.242532:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9a78841d83c0 x1874671476812800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.242540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476812800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.242552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.242554:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828311.242557:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828311.242561:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828311.242571:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828311.242577:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9a78841d8000 x1874671476812928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.242585:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476812928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.242591:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.242593:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828311.242595:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476812928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828311.242599:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828311.242612:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828311.242618:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9a78841db480 x1874671476813056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.242626:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476813056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.242634:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.242636:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828311.242639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828311.242643:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828311.242653:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828311.242659:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9a78841d9e00 x1874671476813184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828311.242666:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476813184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828311.242674:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828311.242675:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828311.242677:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828312.266099:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828313.290078:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828314.313982:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828315.338111:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828316.362018:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.362051:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828316.362058:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828316.362062:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828316.362073:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2683 00000100:00080000:2.0:1787828316.362115:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.362125:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828316.362189:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476813312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828316.362220:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.362224:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828316.362226:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828316.362230:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2683 00000100:00080000:2.0:1787828316.362235:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.362239:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828316.362252:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828316.362266:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476813312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828316.362267:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476813440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828316.362295:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.362299:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828316.362301:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828316.362306:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2683 00000100:00080000:2.0:1787828316.362309:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.362312:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828316.362317:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828316.362339:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476813568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828316.362349:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.362353:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828316.362355:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828316.362360:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2683 00000100:00080000:2.0:1787828316.362363:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.362366:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828316.362379:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828316.362401:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476813696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828316.362403:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828316.362419:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828316.362423:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828316.362426:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828316.362432:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2683 00000100:00080000:2.0:1787828316.362462:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828316.362468:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828316.362474:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828316.362489:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476813824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828316.362496:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9a78841d8780 x1874671476813312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.362519:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476813312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.362532:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.362537:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828316.362548:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476813312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828316.362589:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828316.362608:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828316.362617:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9a78841dbc00 x1874671476813440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.362625:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476813440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.362631:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.362634:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828316.362636:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828316.362640:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828316.362650:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828316.362656:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9a78841da1c0 x1874671476813568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.362667:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476813568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.362676:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.362680:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828316.362683:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828316.362690:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828316.362712:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828316.362724:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9a78841db0c0 x1874671476813696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.362736:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476813696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.362744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.362746:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828316.362748:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828316.362753:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828316.362767:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828316.362775:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9a78841d9a40 x1874671476813824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828316.362793:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476813824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828316.362800:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828316.362802:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828316.362803:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476813824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828317.386072:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828318.409982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828319.434051:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828320.457983:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828321.482041:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828321.482066:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828321.482079:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828321.482085:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828321.482100:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2688 00000100:00080000:2.0:1787828321.482115:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828321.482125:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828321.482186:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476813952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828321.482225:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828321.482230:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828321.482233:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828321.482239:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2688 00000100:00080000:2.0:1787828321.482246:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828321.482251:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828321.482282:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828321.482287:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476813952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828321.482304:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476814080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828321.482321:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828321.482326:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828321.482328:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828321.482336:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2688 00000100:00080000:2.0:1787828321.482354:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828321.482360:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828321.482367:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828321.482384:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476814208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828321.482400:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828321.482406:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828321.482409:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828321.482417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787828321.482418:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2688 00000100:00080000:2.0:1787828321.482435:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828321.482443:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828321.482452:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828321.482465:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828321.482488:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476814336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828321.482508:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828321.482513:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828321.482516:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828321.482526:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2688 00000100:00100000:0.0:1787828321.482547:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814336:192.168.203.140@tcp:8:kworker.0 00000100:00080000:2.0:1787828321.482552:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828321.482564:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828321.482574:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828321.482614:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476814464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828321.482675:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828322.506231:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828323.530041:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828324.554319:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828324.554402:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787828324.554452:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828324.554471:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828324.554520:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828324.554531:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828324.554545:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828324.554567:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787828324.554601:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828324] req@ffff9a78841dbc00 x1874671476814336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828324.554641:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476814336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828324.554658:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828324.554665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828324.554671:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828324.554680:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828324] req@ffff9a78841db0c0 x1874671476814080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828324.554696:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476814080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828324.554705:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828324.554710:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828324.554714:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828324.554722:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828324] req@ffff9a78841da1c0 x1874671476814208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828324.554737:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476814208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828324.554747:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828324.554751:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828324.554756:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828324.554764:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828324] req@ffff9a78841d9a40 x1874671476813952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828324.554780:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476813952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828324.554790:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828324.554794:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828324.554799:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476813952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828324.554873:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828324] req@ffff9a78841d8780 x1874671476814464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828324.554890:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476814464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828324.554902:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828324.554907:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828324.554911:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828325.578188:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828326.602076:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828326.602087:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828326.602093:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828326.602106:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2694 00000100:00080000:0.0:1787828326.602118:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828326.602126:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828326.602176:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828326.602194:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476814592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828326.602250:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828326.602255:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828326.602257:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828326.602262:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2694 00000100:00080000:0.0:1787828326.602267:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828326.602273:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828326.602304:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828326.602315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476814592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828326.602325:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476814720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828326.602349:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828326.602365:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828326.602367:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828326.602372:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2694 00000100:00080000:0.0:1787828326.602377:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828326.602383:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828326.602389:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828326.602405:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476814848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828326.602419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828326.602437:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828326.602443:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828326.602445:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828326.602458:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2694 00000100:00080000:0.0:1787828326.602463:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828326.602469:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828326.602477:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828326.602495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9a7890a3da40 x1874671476814592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828326.602497:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476814976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828326.602534:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828326.602541:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828326.602544:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828326.602551:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2694 00000100:00100000:2.0:1787828326.602557:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476814592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828326.602560:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828326.602572:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828326.602592:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787828326.602598:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828326.602608:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828326.602615:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476815104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828326.602622:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476814592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828326.602672:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828326.602700:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828326.602715:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9a7890a3de00 x1874671476814720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828326.602729:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476814720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828326.602738:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828326.602742:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828326.602746:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828326.602753:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828326.602788:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828326.602802:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9a7890a3f480 x1874671476814848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828326.602816:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476814848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828326.602825:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828326.602854:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828326.602858:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828326.602866:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828326.602889:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828326.602903:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9a7890a3c3c0 x1874671476814976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828326.602915:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476814976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828326.602925:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828326.602928:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828326.602931:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476814976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828326.602937:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828326.602956:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828326.602986:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9a7890a3c000 x1874671476815104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828326.602999:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476815104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828326.603014:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828326.603018:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828326.603021:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828327.626183:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828328.650137:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828329.673994:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828330.697995:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828331.722011:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828331.722024:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828331.722030:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828331.722044:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2699 00010000:00010000:3.0:1787828331.722055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828331.722060:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828331.722072:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828331.722120:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476815232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828331.722162:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828331.722170:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828331.722174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828331.722181:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2699 00000100:00080000:1.0:1787828331.722189:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828331.722199:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828331.722234:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828331.722245:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476815232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828331.722272:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476815360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828331.722289:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828331.722293:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828331.722296:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828331.722300:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2699 00000100:00080000:1.0:1787828331.722306:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828331.722317:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828331.722323:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828331.722337:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476815488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828331.722347:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828331.722351:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828331.722353:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828331.722357:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2699 00000100:00080000:1.0:1787828331.722360:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828331.722363:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828331.722368:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828331.722377:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476815616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828331.722386:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828331.722389:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828331.722391:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828331.722395:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2699 00000100:00080000:1.0:1787828331.722399:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828331.722403:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828331.722408:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828331.722417:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476815744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828331.722534:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828331.722636:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9a7890b1c000 x1874671476815232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.722682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476815232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.722709:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.722716:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828331.722736:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476815232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828331.722792:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828331.722914:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828331.722960:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9a7890b1fc00 x1874671476815360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.723035:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476815360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.723078:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.723086:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828331.723179:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828331.723238:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828331.723341:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828331.723390:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9a7890b1f840 x1874671476815488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.723455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476815488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.723466:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.723470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828331.723493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828331.723502:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828331.723564:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828331.723598:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9a7890b1f480 x1874671476815616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.723614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476815616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.723638:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.723642:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828331.723646:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828331.723668:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828331.723706:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828331.723738:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9a7890b1f0c0 x1874671476815744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.723753:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476815744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.723789:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.723803:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828331.723808:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476815744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828332.746040:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828333.770056:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828334.793956:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828335.818001:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828336.842042:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828336.842060:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828336.842068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828336.842084:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2704 00010000:00010000:0.0:1787828336.842092:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828336.842110:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.842132:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828336.842193:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476815872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828336.842228:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828336.842233:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828336.842236:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828336.842241:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2704 00000100:00080000:2.0:1787828336.842245:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.842249:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828336.842261:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828336.842277:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476816000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828336.842293:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828336.842300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476815872:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828336.842301:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828336.842306:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828336.842318:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2704 00000100:00080000:2.0:1787828336.842325:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.842329:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828336.842336:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828336.842361:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476816128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828336.842381:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828336.842388:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828336.842392:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787828336.842418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828336.842420:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2704 00000100:00080000:2.0:1787828336.842433:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.842452:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828336.842464:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828336.842487:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476816256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828336.842505:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828336] req@ffff9a78841d9e00 x1874671476815872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828336.842510:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828336.842523:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828336.842528:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828336.842542:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2704 00000100:00080000:2.0:1787828336.842554:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.842560:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828336.842567:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828336.842577:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476815872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828336.842585:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476816384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828336.842648:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828336.842657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828336.842666:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476815872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828336.842700:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828336.842735:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828336.842752:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828336] req@ffff9a78841db480 x1874671476816000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828336.842769:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476816000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828336.842780:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828336.842785:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828336.842790:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828336.842797:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828336.842817:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828336.842869:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828336] req@ffff9a78841d8000 x1874671476816128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828336.842914:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476816128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828336.842937:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828336.842943:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828336.842950:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828336.842964:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828336.843005:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828336.843028:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828336] req@ffff9a78841d83c0 x1874671476816256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828336.843053:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476816256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828336.843073:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828336.843080:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828336.843087:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828336.843100:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828336.843152:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828336.843175:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828336] req@ffff9a78841da580 x1874671476816384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828336.843212:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476816384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828336.843232:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828336.843239:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828336.843245:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787828337.866034:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828338.890177:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828339.914166:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828340.938050:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828341.962006:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828341.962020:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828341.962026:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828341.962039:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2709 00000100:00080000:1.0:1787828341.962054:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828341.962065:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828341.962114:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476816512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828341.962163:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828341.962171:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828341.962175:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828341.962183:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2709 00000100:00080000:1.0:1787828341.962191:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787828341.962197:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828341.962199:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828341.962216:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828341.962244:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476816640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828341.962252:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476816512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828341.962284:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828341.962289:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828341.962292:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828341.962313:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2709 00000100:00080000:1.0:1787828341.962321:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828341.962326:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828341.962334:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828341.962351:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476816768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828341.962365:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1787828341.962375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828341.962376:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828341.962385:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828341.962392:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2709 00000100:00080000:1.0:1787828341.962398:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828341.962404:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828341.962411:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828341.962428:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476816896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828341.962440:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828341.962445:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828341.962448:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828341.962455:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2709 00000100:00080000:1.0:1787828341.962462:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828341.962471:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828341.962474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9a7890b1c3c0 x1874671476816512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787828341.962478:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828341.962507:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476817024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828341.962529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476816512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.962552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.962559:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828341.962566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476816512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828341.962608:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828341.962659:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828341.962685:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9a7890b1d680 x1874671476816640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828341.962712:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476816640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.962723:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.962728:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828341.962732:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828341.962740:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828341.962767:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828341.962780:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9a7890b1e580 x1874671476816768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828341.962795:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476816768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.962805:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.962808:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828341.962812:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828341.962820:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828341.962918:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828341.962935:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9a7890b1de00 x1874671476816896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828341.962951:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476816896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.962971:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.962975:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828341.962979:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476816896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828341.962987:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828341.963008:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828341.963020:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9a7890b1e940 x1874671476817024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828341.963033:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476817024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.963045:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.963048:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828341.963052:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828342.986154:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828344.010073:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828345.034210:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828346.058036:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828347.082047:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828347.082059:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828347.082064:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828347.082095:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2714 00000100:00080000:1.0:1787828347.082110:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828347.082120:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828347.082130:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828347.082168:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476817152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828347.082211:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828347.082219:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828347.082224:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828347.082232:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2714 00000100:00080000:1.0:1787828347.082240:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828347.082267:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828347.082282:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828347.082283:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476817152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828347.082317:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476817280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828347.082338:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828347.082345:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828347.082348:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828347.082353:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2714 00000800:00000100:0.0:1787828347.082378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828347.082380:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828347.082389:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828347.082400:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828347.082424:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476817408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828347.082443:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828347.082452:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1787828347.082453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff9a7890b1da40 x1874671476817152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828347.082457:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828347.082474:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2714 00000100:00080000:1.0:1787828347.082489:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828347.082495:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828347.082503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476817152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787828347.082504:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828347.082529:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476817536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828347.082534:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828347.082548:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787828347.082554:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828347.082559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476817152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828347.082562:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828347.082570:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828347.082577:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2714 00000100:00080000:1.0:1787828347.082582:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828347.082586:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828347.082592:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828347.082600:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828347.082609:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476817664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828347.082661:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828347.082684:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff9a7890b1e940 x1874671476817280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828347.082700:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476817280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828347.082711:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828347.082715:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828347.082720:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828347.082727:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828347.082748:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828347.082760:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff9a7890b1de00 x1874671476817408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828347.082773:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476817408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828347.082783:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828347.082787:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828347.082791:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828347.082798:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828347.082911:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828347.082932:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff9a7890b1e580 x1874671476817536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828347.082948:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476817536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828347.082960:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828347.082964:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828347.082968:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828347.083007:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828347.083032:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828347.083043:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828347/real 1787828347] req@ffff9a7890b1d680 x1874671476817664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828347.083055:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476817664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828347.083065:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828347.083068:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828347.083071:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828348.106095:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828349.130026:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828350.154104:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828351.177967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828352.201970:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787828352.201986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828352.201998:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828352.202007:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828352.202023:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2719 00000100:00080000:2.0:1787828352.202049:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828352.202060:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828352.202143:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476817792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828352.202209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828352.202215:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828352.202218:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828352.202223:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2719 00000100:00080000:2.0:1787828352.202230:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828352.202237:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828352.202251:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828352.202278:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476817792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828352.202292:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476817920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828352.202324:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828352.202330:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828352.202334:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828352.202341:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2719 00000100:00080000:2.0:1787828352.202347:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787828352.202348:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828352.202360:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828352.202386:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828352.202408:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476818048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828352.202423:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff9a78841d8b40 x1874671476817792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828352.202426:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828352.202433:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828352.202451:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828352.202462:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2719 00000100:00080000:2.0:1787828352.202472:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828352.202483:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828352.202493:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828352.202518:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476817792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828352.202533:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476818176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828352.202554:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828352.202566:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828352.202567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787828352.202579:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:0.0:1787828352.202582:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476817792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828352.202585:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828352.202599:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2719 00000100:00080000:2.0:1787828352.202605:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828352.202612:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828352.202619:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828352.202626:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828352.202643:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476818304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828352.202664:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828352.202691:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff9a78841db840 x1874671476817920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828352.202705:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476817920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828352.202714:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828352.202717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828352.202721:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476817920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828352.202726:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828352.202741:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828352.202750:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff9a78841d92c0 x1874671476818048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828352.202768:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476818048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828352.202779:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828352.202813:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828352.202819:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828352.202856:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828352.202909:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828352.202926:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff9a78841d9680 x1874671476818176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828352.202944:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476818176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828352.202959:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828352.202964:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828352.202968:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828352.202994:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828352.203016:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828352.203026:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828352/real 1787828352] req@ffff9a78841da940 x1874671476818304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828352.203047:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476818304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828352.203062:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828352.203065:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828352.203068:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828353.225943:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828354.250017:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828355.273938:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828356.297980:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828357.322008:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.322033:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828357.322040:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.322053:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2724 00000100:00080000:0.0:1787828357.322067:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828357.322076:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828357.322112:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828357.322126:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476818432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828357.322166:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.322185:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828357.322189:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.322197:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2724 00000100:00080000:0.0:1787828357.322205:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828357.322213:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828357.322229:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828357.322249:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476818432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828357.322254:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476818560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828357.322301:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.322307:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828357.322310:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.322318:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2724 00000100:00080000:0.0:1787828357.322337:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828357.322345:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787828357.322352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787828357.322354:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828357.322376:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476818688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828357.322393:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.322400:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828357.322403:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.322410:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2724 00000100:00080000:0.0:1787828357.322416:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828357.322423:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828357.322424:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff9a7890a3fc00 x1874671476818432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828357.322431:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828357.322449:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476818816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828357.322455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476818432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828357.322468:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828357.322474:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828357.322476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828357.322477:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828357.322484:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2724 00000100:00080000:2.0:1787828357.322484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787828357.322491:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828357.322492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476818432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828357.322497:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828357.322513:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828357.322530:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476818944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828357.322553:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828357.322602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828357.322629:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff9a7890a3c000 x1874671476818560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828357.322647:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476818560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828357.322658:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828357.322662:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828357.322667:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828357.322675:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828357.322702:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828357.322716:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff9a7890a3c3c0 x1874671476818688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828357.322730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476818688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828357.322740:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828357.322775:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828357.322779:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828357.322796:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828357.322820:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828357.322875:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff9a7890a3f480 x1874671476818816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828357.322891:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476818816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828357.322903:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828357.322907:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828357.322911:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828357.322931:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828357.322956:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828357.322969:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828357/real 1787828357] req@ffff9a7890a3de00 x1874671476818944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828357.322983:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476818944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828357.322995:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828357.322998:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828357.323002:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476818944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828358.345979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828359.369982:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828360.393946:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828361.417969:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828362.441940:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828362.441948:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828362.441951:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828362.441960:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2729 00000100:00080000:1.0:1787828362.441970:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828362.441976:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828362.442001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828362.442015:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476819072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828362.442037:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828362.442040:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828362.442042:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828362.442044:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2729 00000100:00080000:1.0:1787828362.442046:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828362.442049:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828362.442062:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828362.442072:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476819200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828362.442080:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476819072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828362.442094:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828362.442097:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828362.442098:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828362.442103:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2729 00000100:00080000:1.0:1787828362.442105:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828362.442107:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828362.442111:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828362.442120:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476819328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828362.442127:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828362.442129:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828362.442131:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828362.442133:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2729 00000100:00080000:1.0:1787828362.442136:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828362.442137:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828362.442141:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787828362.442142:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828362.442152:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476819456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828362.442161:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828362.442163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828362.442165:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828362.442167:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2729 00000100:00080000:1.0:1787828362.442171:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828362.442173:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828362.442176:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828362.442185:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476819584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828362.442195:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff9a7890b1c3c0 x1874671476819072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.442220:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476819072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.442232:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.442237:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828362.442242:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476819072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828362.442276:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828362.442307:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828362.442317:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff9a7890b1f0c0 x1874671476819200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.442326:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476819200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.442332:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.442335:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828362.442337:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828362.442346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828362.442358:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828362.442365:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff9a7890b1f480 x1874671476819328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.442372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476819328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.442378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.442379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828362.442382:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828362.442386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828362.442397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828362.442403:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff9a7890b1f840 x1874671476819456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.442410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476819456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.442416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.442418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828362.442420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828362.442423:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828362.442432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828362.442437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828362/real 1787828362] req@ffff9a7890b1fc00 x1874671476819584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828362.442445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476819584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828362.442451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828362.442452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828362.442454:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828363.466056:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828364.490087:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828365.514145:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828366.538070:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828367.561994:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828367.562011:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828367.562018:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828367.562036:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2735 00000100:00080000:1.0:1787828367.562054:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828367.562066:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828367.562118:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828367.562142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476819712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828367.562209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828367.562215:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828367.562218:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828367.562223:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2735 00000100:00080000:1.0:1787828367.562229:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828367.562234:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828367.562249:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828367.562273:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476819840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828367.562302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476819712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828367.562318:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828367.562325:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828367.562327:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828367.562334:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2735 00000100:00080000:1.0:1787828367.562339:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828367.562344:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828367.562350:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828367.562368:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476819968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828367.562381:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828367.562386:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828367.562388:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828367.562392:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2735 00000100:00080000:1.0:1787828367.562410:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787828367.562421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828367.562422:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828367.562440:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828367.562467:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476820096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828367.562487:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828367.562493:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:2.0:1787828367.562502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828367/real 1787828367] req@ffff9a7890b1c000 x1874671476819712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828367.562512:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828367.562527:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2735 00000100:00080000:1.0:1787828367.562535:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828367.562536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476819712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828367.562549:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828367.562567:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787828367.562569:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828367.562583:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828367.562590:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476820224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828367.562601:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476819712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828367.562658:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828367.562743:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828367.562765:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828367/real 1787828367] req@ffff9a7890b1fc00 x1874671476819840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828367.562782:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476819840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828367.562793:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828367.562798:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828367.562802:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828367.562811:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828367.562834:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828367.562891:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828367/real 1787828367] req@ffff9a7890b1f840 x1874671476819968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828367.562910:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476819968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828367.562930:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828367.562934:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828367.562938:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476819968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828367.562946:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828367.562971:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828367.562984:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828367/real 1787828367] req@ffff9a7890b1f480 x1874671476820096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828367.562999:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476820096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828367.563012:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828367.563016:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828367.563020:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828367.563027:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828367.563046:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828367.563057:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828367/real 1787828367] req@ffff9a7890b1f0c0 x1874671476820224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828367.563071:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476820224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828367.563082:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828367.563086:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828367.563091:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828368.586095:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828369.610127:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828370.633995:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828371.658014:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828372.681966:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828372.681978:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828372.681983:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828372.681994:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2740 00000100:00080000:1.0:1787828372.682008:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828372.682021:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828372.682070:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476820352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828372.682110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787828372.682110:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828372.682122:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828372.682153:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828372.682162:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2740 00000100:00080000:1.0:1787828372.682168:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828372.682177:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828372.682189:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828372.682214:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476820352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828372.682215:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476820480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828372.682242:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828372.682246:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828372.682247:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828372.682254:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2740 00000100:00080000:1.0:1787828372.682258:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828372.682263:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828372.682268:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828372.682283:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476820608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828372.682291:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828372.682294:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828372.682295:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828372.682298:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2740 00000100:00080000:1.0:1787828372.682312:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828372.682318:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828372.682328:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787828372.682330:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828372.682349:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476820736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828372.682357:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828372.682360:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828372.682361:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828372.682365:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2740 00000100:00080000:1.0:1787828372.682368:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828372.682372:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828372.682378:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828372.682396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828372/real 1787828372] req@ffff9a7890b1c3c0 x1874671476820352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828372.682399:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476820864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828372.682448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476820352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828372.682462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828372.682467:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828372.682482:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476820352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828372.682515:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828372.682536:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828372.682548:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828372/real 1787828372] req@ffff9a7890b1d680 x1874671476820480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828372.682558:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476820480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828372.682566:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828372.682569:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828372.682572:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828372.682577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828372.682589:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828372.682622:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828372/real 1787828372] req@ffff9a7890b1e580 x1874671476820608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828372.682635:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476820608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828372.682660:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828372.682664:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828372.682668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828372.682676:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828372.682698:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828372.682707:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828372/real 1787828372] req@ffff9a7890b1de00 x1874671476820736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828372.682720:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476820736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828372.682728:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828372.682731:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828372.682733:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828372.682737:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828372.682749:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828372.682755:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828372/real 1787828372] req@ffff9a7890b1e940 x1874671476820864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828372.682763:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476820864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828372.682770:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828372.682772:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828372.682773:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476820864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828373.706100:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828374.730032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828375.754053:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828376.778106:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828377.801952:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828377.801966:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828377.801971:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828377.801985:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2745 00000100:00080000:1.0:1787828377.801999:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828377.802009:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828377.802061:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476820992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828377.802106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828377.802111:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828377.802114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828377.802123:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2745 00000100:00080000:1.0:1787828377.802127:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828377.802132:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828377.802145:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828377.802186:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828377.802186:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476821120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828377.802233:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476820992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828377.802237:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828377.802258:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828377.802267:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828377.802273:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2745 00000100:00080000:1.0:1787828377.802278:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828377.802282:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828377.802290:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828377.802310:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476821248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828377.802324:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828377.802332:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828377.802337:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828377.802348:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2745 00000100:00080000:1.0:1787828377.802390:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828377.802400:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787828377.802414:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787828377.802415:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828377.802448:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476821376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828377.802463:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828377.802467:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828377.802470:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828377.802487:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2745 00000100:00080000:1.0:1787828377.802492:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828377.802497:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828377.802506:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828377.802525:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476821504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828377.802545:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828377/real 1787828377] req@ffff9a7890b1da40 x1874671476820992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828377.802586:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476820992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828377.802633:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828377.802642:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828377.802647:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476820992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828377.802693:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828377.802742:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828377.802771:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828377/real 1787828377] req@ffff9a7890b1e940 x1874671476821120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828377.802820:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476821120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828377.802893:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828377.802897:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828377.802902:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828377.802910:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828377.802940:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828377.802955:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828377/real 1787828377] req@ffff9a7890b1de00 x1874671476821248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828377.802968:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476821248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828377.802977:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828377.802981:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828377.802985:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828377.802991:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828377.803024:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828377.803035:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828377/real 1787828377] req@ffff9a7890b1e580 x1874671476821376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828377.803048:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476821376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828377.803064:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828377.803071:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828377.803076:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828377.803087:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828377.803120:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828377.803140:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828377/real 1787828377] req@ffff9a7890b1d680 x1874671476821504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828377.803167:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476821504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828377.803198:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828377.803204:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828377.803211:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828378.826110:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828379.849991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828380.874058:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828381.898111:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828382.921929:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828382.921940:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828382.921944:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828382.921956:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2750 00000100:00080000:2.0:1787828382.921967:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828382.921969:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828382.921975:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828382.922011:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476821632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828382.922037:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828382.922040:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828382.922041:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828382.922045:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2750 00000100:00080000:2.0:1787828382.922047:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828382.922049:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828382.922057:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828382.922069:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476821760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828382.922078:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828382.922080:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476821632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828382.922082:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828382.922086:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828382.922094:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2750 00000100:00080000:2.0:1787828382.922097:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828382.922099:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828382.922103:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828382.922113:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476821888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828382.922121:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828382.922126:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828382.922128:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828382.922132:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2750 00000100:00080000:2.0:1787828382.922137:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828382.922142:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828382.922147:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828382.922149:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828382.922166:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476822016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828382.922177:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828382.922182:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828382.922184:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828382.922191:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828382/real 1787828382] req@ffff9a78841da940 x1874671476821632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828382.922192:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2750 00000100:00080000:2.0:1787828382.922200:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828382.922209:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828382.922214:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476821632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787828382.922218:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787828382.922229:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828382.922233:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828382.922240:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476821632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828382.922240:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476822144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828382.922274:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828382.922300:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828382.922324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828382/real 1787828382] req@ffff9a78841d9680 x1874671476821760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828382.922334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476821760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828382.922341:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828382.922343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828382.922346:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828382.922351:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828382.922372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828382.922379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828382/real 1787828382] req@ffff9a78841d92c0 x1874671476821888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828382.922393:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476821888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828382.922398:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828382.922400:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828382.922402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476821888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828382.922407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828382.922418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828382.922424:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828382/real 1787828382] req@ffff9a78841db840 x1874671476822016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828382.922432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476822016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828382.922439:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828382.922441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828382.922443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828382.922446:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828382.922456:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828382.922462:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828382/real 1787828382] req@ffff9a78841d8b40 x1874671476822144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828382.922470:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476822144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828382.922481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828382.922483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828382.922485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828383.946134:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828384.969976:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828385.994017:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828387.018034:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828388.041944:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828388.041962:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828388.041971:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828388.041987:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2755 00000100:00080000:2.0:1787828388.042002:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828388.042013:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828388.042077:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476822272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828388.042078:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828388.042148:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828388.042157:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828388.042162:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828388.042172:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2755 00000100:00080000:2.0:1787828388.042182:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828388.042193:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828388.042210:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828388.042212:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476822272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828388.042244:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476822400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828388.042280:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828388.042288:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828388.042291:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828388.042300:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2755 00000100:00080000:2.0:1787828388.042308:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828388.042316:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828388.042327:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828388.042357:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476822528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828388.042378:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828388.042386:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828388.042406:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828388.042414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822400:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787828388.042423:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2755 00000100:00080000:2.0:1787828388.042435:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828388.042441:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828388.042452:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828388.042475:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476822656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828388.042492:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828388.042493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822528:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787828388.042504:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828388.042509:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828388.042518:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2755 00000100:00080000:2.0:1787828388.042525:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828388.042531:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828388.042568:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828388.042587:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828388.042594:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476822784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828388.042657:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828389.066010:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828390.090081:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828391.114045:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787828391.114158:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787828391.114202:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787828391.114218:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828391.114261:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828391.114272:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828391.114280:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828391.114288:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787828391.114368:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828391] req@ffff9a78841d9e00 x1874671476822784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828391.114424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476822784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828391.114450:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828391.114460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828391.114469:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822784:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828391.114484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828391] req@ffff9a78841db480 x1874671476822656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828391.114508:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476822656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828391.114549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828391.114558:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828391.114564:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828391.114578:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828391] req@ffff9a78841d83c0 x1874671476822400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828391.114603:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476822400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828391.114618:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828391.114625:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828391.114631:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828391.114644:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828391] req@ffff9a78841d8000 x1874671476822528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828391.114667:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476822528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828391.114682:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828391.114688:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828391.114694:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476822528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828391.114707:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828391] req@ffff9a78841da580 x1874671476822272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828391.114730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476822272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828391.114745:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828391.114752:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828391.114758:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476822272:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787828392.138051:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828393.162129:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828393.162134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828393.162177:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828393.162188:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828393.162213:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2760 00000100:00080000:1.0:1787828393.162237:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828393.162252:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828393.162338:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476822912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828393.162392:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828393.162401:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828393.162406:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828393.162414:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2760 00000100:00080000:1.0:1787828393.162423:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828393.162435:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828393.162467:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828393.162478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476822912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828393.162536:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476823040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828393.162563:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828393.162570:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828393.162575:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828393.162584:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2760 00000100:00080000:1.0:1787828393.162619:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828393.162632:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828393.162645:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787828393.162649:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828393.162685:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476823168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828393.162705:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828393.162712:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828393.162716:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828393.162726:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2760 00000100:00080000:1.0:1787828393.162736:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828393.162745:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828393.162756:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828393.162779:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff9a7890b1c3c0 x1874671476822912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828393.162786:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476823296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828393.162888:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828393.162895:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828393.162900:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828393.162910:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2760 00000100:00080000:1.0:1787828393.162919:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828393.162935:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828393.162948:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828393.162957:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476822912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828393.162983:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476823424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828393.162990:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828393.163011:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828393.163027:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476822912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828393.163080:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828393.163133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828393.163162:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff9a7890b1f0c0 x1874671476823040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828393.163205:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476823040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828393.163224:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828393.163231:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828393.163238:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828393.163252:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828393.163296:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828393.163323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff9a7890b1f480 x1874671476823168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828393.163350:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476823168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828393.163367:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828393.163373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828393.163377:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828393.163391:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828393.163444:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828393.163468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff9a7890b1f840 x1874671476823296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828393.163492:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476823296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828393.163558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828393.163565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828393.163572:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828393.163585:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828393.163629:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828393.163653:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828393/real 1787828393] req@ffff9a7890b1fc00 x1874671476823424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828393.163678:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476823424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828393.163698:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828393.163706:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828393.163712:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828394.186070:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828395.210156:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828396.234074:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828397.258027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828398.282025:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828398.282040:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828398.282047:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828398.282063:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2765 00000100:00080000:2.0:1787828398.282079:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828398.282091:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828398.282177:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828398.282180:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476823552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828398.282224:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828398.282229:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828398.282232:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828398.282237:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2765 00000100:00080000:2.0:1787828398.282244:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828398.282249:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828398.282263:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828398.282267:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476823552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828398.282282:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476823680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828398.282315:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828398.282320:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828398.282322:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828398.282327:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2765 00000100:00080000:2.0:1787828398.282332:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828398.282337:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828398.282343:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828398.282360:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476823808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828398.282376:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828398.282380:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828398.282383:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828398.282387:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2765 00000100:00080000:2.0:1787828398.282393:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828398.282396:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828398.282402:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828398.282411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828398.282419:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476823936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828398.282442:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828398.282446:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828398.282449:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828398.282454:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2765 00000100:00080000:2.0:1787828398.282458:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828398.282484:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828398.282490:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828398.282537:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476824064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828398.282552:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff9a78841d8780 x1874671476823552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828398.282580:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476823552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828398.282602:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828398.282607:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828398.282612:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476823552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828398.282645:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828398.282672:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828398.282699:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff9a78841d9a40 x1874671476823680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828398.282718:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476823680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828398.282759:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828398.282765:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828398.282772:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828398.282784:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828398.282823:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828398.282912:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff9a78841da1c0 x1874671476823808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828398.282939:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476823808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828398.282953:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828398.282957:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828398.282962:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828398.282970:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828398.283001:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828398.283015:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff9a78841db0c0 x1874671476823936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828398.283029:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476823936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828398.283040:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828398.283043:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828398.283048:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476823936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828398.283054:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828398.283071:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828398.283082:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828398/real 1787828398] req@ffff9a78841dbc00 x1874671476824064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828398.283094:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476824064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828398.283115:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828398.283118:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828398.283121:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828399.305997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828400.330047:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828401.354022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828402.378038:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828403.401927:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828403.401954:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828403.401958:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828403.401968:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2770 00010000:00010000:1.0:1787828403.401978:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828403.401992:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828403.402002:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828403.402046:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476824192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828403.402084:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828403.402088:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828403.402090:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828403.402095:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2770 00000100:00080000:3.0:1787828403.402100:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828403.402106:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828403.402117:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828403.402136:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476824192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828403.402139:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476824320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828403.402158:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828403.402163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828403.402165:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828403.402170:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2770 00000100:00080000:3.0:1787828403.402175:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828403.402179:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828403.402185:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828403.402214:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476824448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828403.402220:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828403.402228:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828403.402231:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828403.402232:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828403.402237:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2770 00000100:00080000:3.0:1787828403.402240:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828403.402242:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828403.402246:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828403.402256:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476824576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828403.402262:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828403.402265:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff9a78a6777480 x1874671476824192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828403.402266:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828403.402270:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828403.402277:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2770 00000100:00080000:3.0:1787828403.402282:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828403.402285:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828403.402289:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828403.402293:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476824192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828403.402317:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476824704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828403.402321:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828403.402333:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828403.402340:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476824192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828403.402363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828403.402383:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828403.402393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff9a78a6774000 x1874671476824320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828403.402403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476824320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828403.402409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828403.402412:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828403.402414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828403.402418:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828403.402470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828403.402480:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff9a78a6775a40 x1874671476824448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828403.402493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476824448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828403.402499:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828403.402501:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828403.402503:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828403.402507:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828403.402521:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828403.402528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff9a78a67752c0 x1874671476824576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828403.402535:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476824576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828403.402541:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828403.402543:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828403.402545:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828403.402549:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828403.402557:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828403.402562:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828403/real 1787828403] req@ffff9a78a6774b40 x1874671476824704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828403.402571:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476824704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828403.402587:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828403.402590:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828403.402594:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828404.425985:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828405.449940:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828406.474015:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828407.497988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828408.521984:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828408.521999:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828408.522006:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828408.522020:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2776 00000100:00080000:0.0:1787828408.522035:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828408.522058:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828408.522107:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476824832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828408.522115:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828408.522152:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828408.522159:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828408.522162:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828408.522168:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2776 00000100:00080000:0.0:1787828408.522173:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828408.522179:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828408.522192:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828408.522212:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476824832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828408.522223:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476824960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828408.522263:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828408.522271:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828408.522275:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828408.522285:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2776 00000100:00080000:0.0:1787828408.522319:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787828408.522328:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828408.522334:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828408.522346:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828408.522368:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476825088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828408.522382:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828408.522386:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828408.522389:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828408.522427:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2776 00000100:00080000:0.0:1787828408.522434:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828408.522440:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828408.522449:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828408.522454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828408/real 1787828408] req@ffff9a7890a3da40 x1874671476824832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828408.522472:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476825216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828408.522498:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828408.522502:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828408.522505:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828408.522512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476824832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828408.522516:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2776 00000100:00080000:0.0:1787828408.522528:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828408.522540:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:1.0:1787828408.522554:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828408.522564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:0.0:1787828408.522565:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828408.522575:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476824832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828408.522588:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476825344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828408.522633:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828408.522673:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828408.522692:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828408/real 1787828408] req@ffff9a7890a3de00 x1874671476824960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828408.522707:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476824960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828408.522718:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828408.522723:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828408.522727:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476824960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828408.522738:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828408.522758:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828408.522769:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828408/real 1787828408] req@ffff9a7890a3f480 x1874671476825088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828408.522784:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476825088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828408.522793:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828408.522797:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828408.522800:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828408.522807:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828408.522825:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828408.522880:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828408/real 1787828408] req@ffff9a7890a3c3c0 x1874671476825216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828408.522896:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476825216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828408.522908:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828408.522912:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828408.522916:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828408.522923:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828408.522946:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828408.522957:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828408/real 1787828408] req@ffff9a7890a3c000 x1874671476825344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828408.522972:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476825344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828408.522983:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828408.522986:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828408.522990:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828409.545976:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828410.570051:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828411.594098:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828412.618063:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828413.641941:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828413.641953:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828413.641958:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828413.641971:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2781 00000100:00080000:0.0:1787828413.641983:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828413.641991:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828413.642048:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476825472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828413.642049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828413.642083:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828413.642088:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828413.642090:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828413.642094:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2781 00000100:00080000:0.0:1787828413.642098:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828413.642101:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828413.642111:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828413.642125:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476825600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828413.642136:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828413.642140:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828413.642142:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828413.642147:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2781 00000100:00080000:0.0:1787828413.642150:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828413.642153:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828413.642157:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828413.642169:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476825728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828413.642179:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828413.642182:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828413.642184:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828413.642189:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2781 00000100:00080000:0.0:1787828413.642193:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828413.642196:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828413.642218:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828413.642225:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476825472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828413.642236:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476825856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828413.642249:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828413.642253:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828413.642254:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828413.642261:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2781 00000100:00080000:0.0:1787828413.642265:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828413.642270:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828413.642275:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828413.642287:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476825984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828413.642338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828413.642427:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828413/real 1787828413] req@ffff9a7890a3fc00 x1874671476825472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828413.642445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476825472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828413.642456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828413.642460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828413.642463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476825472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828413.642480:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828413.642497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828413.642506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828413/real 1787828413] req@ffff9a7890a3c000 x1874671476825600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828413.642520:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476825600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828413.642525:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828413.642527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828413.642529:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828413.642546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828413.642557:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828413.642563:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828413/real 1787828413] req@ffff9a7890a3c3c0 x1874671476825728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828413.642571:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476825728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828413.642576:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828413.642578:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828413.642580:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828413.642609:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828413.642633:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828413.642639:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828413/real 1787828413] req@ffff9a7890a3f480 x1874671476825856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828413.642646:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476825856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828413.642659:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828413.642662:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828413.642664:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828413.642668:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828413.642678:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828413.642684:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828413/real 1787828413] req@ffff9a7890a3de00 x1874671476825984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828413.642692:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476825984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828413.642699:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828413.642701:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828413.642703:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476825984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828414.666016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828415.690162:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828416.713971:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828417.738053:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828418.761922:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828418.761936:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828418.761941:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828418.761955:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2786 00000100:00080000:3.0:1787828418.761979:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828418.761989:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828418.762044:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476826112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828418.762087:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828418.762091:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828418.762093:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828418.762098:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2786 00000100:00080000:3.0:1787828418.762103:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828418.762107:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828418.762128:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828418.762146:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476826240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828418.762165:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476826112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828418.762165:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828418.762175:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828418.762180:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828418.762186:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2786 00000100:00080000:3.0:1787828418.762190:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828418.762193:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828418.762199:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828418.762214:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476826368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828418.762233:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828418.762236:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828418.762238:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828418.762242:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2786 00000100:00080000:3.0:1787828418.762247:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828418.762251:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828418.762256:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787828418.762262:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828418.762271:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476826496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828418.762283:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828418.762287:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828418.762289:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828418.762293:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2786 00000100:00080000:3.0:1787828418.762296:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828418.762299:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828418.762304:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828418.762314:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476826624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828418.762371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828418/real 1787828418] req@ffff9a78a6776580 x1874671476826112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828418.762396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476826112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828418.762418:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828418.762423:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828418.762430:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476826112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828418.762463:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826240:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787828418.762508:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:1.0:1787828418.762516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828418.762540:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828418/real 1787828418] req@ffff9a78a6776940 x1874671476826240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828418.762554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476826240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828418.762563:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828418.762567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828418.762571:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828418.762577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828418.762594:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828418.762603:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828418/real 1787828418] req@ffff9a78a67770c0 x1874671476826368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828418.762615:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476826368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828418.762624:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828418.762627:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828418.762632:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828418.762638:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828418.762655:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828418.762665:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828418/real 1787828418] req@ffff9a78a6774780 x1874671476826496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828418.762676:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476826496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828418.762687:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828418.762690:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828418.762693:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828418.762716:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828418.762732:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828418.762741:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828418/real 1787828418] req@ffff9a78a67761c0 x1874671476826624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828418.762752:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476826624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828418.762761:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828418.762763:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828418.762766:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828419.785950:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828420.809994:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828421.834042:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828422.857930:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828423.881962:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828423.881971:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828423.881975:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828423.881985:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2791 00000100:00080000:0.0:1787828423.882006:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828423.882014:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828423.882031:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828423.882068:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476826752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828423.882095:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828423.882098:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828423.882099:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828423.882102:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2791 00000100:00080000:0.0:1787828423.882105:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828423.882116:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828423.882125:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828423.882156:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476826880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828423.882157:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476826752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828423.882170:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828423.882173:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828423.882175:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828423.882178:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2791 00000100:00080000:0.0:1787828423.882180:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828423.882183:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828423.882187:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828423.882197:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476827008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828423.882204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828423.882207:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828423.882208:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828423.882211:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2791 00000100:00080000:0.0:1787828423.882220:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828423.882222:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828423.882226:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828423.882234:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476827136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828423.882243:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:1.0:1787828423.882244:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828423.882247:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828423.882249:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828423.882252:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2791 00000100:00080000:0.0:1787828423.882256:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828423.882258:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828423.882262:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828423.882269:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476827264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828423.882311:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828423/real 1787828423] req@ffff9a7890a3da40 x1874671476826752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828423.882330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476826752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828423.882343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828423.882348:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828423.882354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476826752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828423.882382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828423.882403:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828423.882413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828423/real 1787828423] req@ffff9a7890a3de00 x1874671476826880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828423.882422:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476826880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828423.882428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828423.882430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828423.882432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476826880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828423.882436:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828423.882447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828423.882452:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828423/real 1787828423] req@ffff9a7890a3f480 x1874671476827008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828423.882465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476827008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828423.882470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828423.882472:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828423.882474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828423.882477:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828423.882488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828423.882494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828423/real 1787828423] req@ffff9a7890a3c3c0 x1874671476827136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828423.882501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476827136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828423.882508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828423.882509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828423.882511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828423.882515:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828423.882524:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828423.882529:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828423/real 1787828423] req@ffff9a7890a3c000 x1874671476827264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828423.882537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476827264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828423.882548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828423.882550:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828423.882552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828424.906051:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828425.930020:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828426.954072:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828427.978075:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828429.001930:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828429.001944:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828429.001950:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828429.001965:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2796 00000100:00080000:0.0:1787828429.001981:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828429.001990:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828429.002013:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828429.002046:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476827392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828429.002104:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828429.002110:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828429.002113:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828429.002118:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2796 00000100:00080000:0.0:1787828429.002123:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828429.002128:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828429.002145:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828429.002165:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476827520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828429.002182:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476827392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828429.002185:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828429.002199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828429.002203:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828429.002210:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2796 00000100:00080000:0.0:1787828429.002214:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828429.002218:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828429.002227:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828429.002244:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476827648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828429.002287:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828429.002292:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828429.002295:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828429.002300:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2796 00000100:00080000:0.0:1787828429.002310:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828429.002315:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828429.002323:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:1.0:1787828429.002331:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828429.002344:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476827776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828429.002359:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828429.002364:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828429.002367:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828429.002371:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2796 00000100:00080000:0.0:1787828429.002376:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828429.002381:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828429.002387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828429.002404:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476827904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828429.002419:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff9a7890a3fc00 x1874671476827392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828429.002452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476827392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828429.002471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828429.002480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828429.002488:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476827392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828429.002536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828429.002602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828429.002623:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff9a7890a3c000 x1874671476827520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828429.002641:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476827520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828429.002653:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828429.002657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828429.002663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828429.002672:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828429.002718:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828429.002727:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff9a7890a3c3c0 x1874671476827648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828429.002737:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476827648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828429.002744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828429.002746:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828429.002749:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828429.002753:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828429.002768:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828429.002778:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff9a7890a3f480 x1874671476827776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828429.002794:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476827776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828429.002808:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828429.002813:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828429.002817:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828429.002826:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828429.002889:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828429.002910:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828429/real 1787828429] req@ffff9a7890a3de00 x1874671476827904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828429.002928:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476827904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828429.002939:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828429.002942:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828429.002945:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476827904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828430.026102:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828431.050066:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828432.074130:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828433.098027:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828434.122029:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828434.122042:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:2.0:1787828434.122047:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828434.122048:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828434.122067:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2801 00000100:00080000:3.0:1787828434.122083:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828434.122105:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828434.122153:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476828032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828434.122188:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828434.122192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828434.122195:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828434.122200:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2801 00000100:00080000:3.0:1787828434.122204:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828434.122208:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828434.122254:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828434.122285:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476828160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828434.122292:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828434.122294:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828434.122296:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828434.122299:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2801 00000100:00080000:3.0:1787828434.122313:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828434.122316:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828434.122320:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828434.122329:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476828288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828434.122335:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476828032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828434.122338:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828434.122343:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828434.122344:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828434.122348:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2801 00000100:00080000:3.0:1787828434.122350:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828434.122352:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828434.122355:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828434.122362:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476828416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828434.122368:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828434.122370:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828434.122371:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828434.122374:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2801 00000100:00080000:3.0:1787828434.122376:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828434.122378:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828434.122381:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828434.122388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476828544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828434.122403:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828434.122451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9a78a6776d00 x1874671476828032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828434.122465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476828032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828434.122475:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828434.122478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828434.122484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476828032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828434.122491:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828434.122506:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828434.122514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9a78a6775680 x1874671476828160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828434.122526:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476828160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828434.122531:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828434.122533:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828434.122535:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828434.122555:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828434.122565:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828434.122575:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9a78a6775e00 x1874671476828288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828434.122588:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476828288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828434.122596:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828434.122600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828434.122603:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828434.122610:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828434.122630:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828434.122639:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9a78a6777840 x1874671476828416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828434.122647:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476828416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828434.122658:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828434.122660:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828434.122662:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828434.122667:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828434.122687:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828434.122693:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828434/real 1787828434] req@ffff9a78a6774f00 x1874671476828544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828434.122700:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476828544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828434.122706:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828434.122708:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828434.122710:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828435.146090:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828436.170089:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828437.194090:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828438.218092:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828439.241987:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828439.242003:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828439.242011:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828439.242026:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2806 00000100:00080000:3.0:1787828439.242039:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828439.242048:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828439.242093:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476828672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828439.242132:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828439.242139:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828439.242143:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828439.242150:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2806 00000100:00080000:3.0:1787828439.242157:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828439.242165:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828439.242203:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828439.242225:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476828672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828439.242227:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476828800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828439.242246:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828439.242251:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828439.242254:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828439.242260:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2806 00000100:00080000:3.0:1787828439.242266:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828439.242272:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828439.242279:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828439.242298:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476828928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828439.242310:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828439.242314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828439.242319:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828439.242322:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828439.242328:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2806 00000100:00080000:3.0:1787828439.242348:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828439.242355:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828439.242364:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828439.242381:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476829056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828439.242390:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9a78a6777c00 x1874671476828672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828439.242403:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828439.242409:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828439.242412:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828439.242419:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2806 00000100:00100000:1.0:1787828439.242425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476828672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828439.242425:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828439.242433:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828439.242441:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:1.0:1787828439.242445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828439.242453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828439.242457:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476829184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828439.242461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476828672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828439.242495:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828439.242535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828439.242560:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9a78a67743c0 x1874671476828800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828439.242590:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476828800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828439.242596:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828439.242616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828439.242624:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828439.242632:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828439.242642:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828439.242672:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828439.242687:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9a78a6774f00 x1874671476828928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828439.242701:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476828928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828439.242710:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828439.242714:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828439.242718:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476828928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828439.242724:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828439.242742:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828439.242753:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9a78a6777840 x1874671476829056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828439.242766:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476829056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828439.242776:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828439.242779:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828439.242783:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828439.242788:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828439.242805:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828439.242815:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828439/real 1787828439] req@ffff9a78a6775e00 x1874671476829184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828439.242870:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476829184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828439.242882:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828439.242886:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828439.242890:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828440.266030:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828441.290055:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828442.314032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828443.338093:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828444.361985:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828444.361998:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828444.362003:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828444.362016:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2811 00000100:00080000:3.0:1787828444.362030:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828444.362050:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828444.362096:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476829312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828444.362134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828444.362187:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828444.362190:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828444.362195:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2811 00000100:00080000:3.0:1787828444.362199:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828444.362205:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828444.362215:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828444.362229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476829312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828444.362235:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476829440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828444.362262:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828444.362266:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828444.362268:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828444.362272:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2811 00000100:00080000:3.0:1787828444.362276:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828444.362279:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828444.362284:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828444.362298:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476829568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828444.362308:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828444.362312:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:1.0:1787828444.362318:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828444.362318:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828444.362327:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2811 00000100:00080000:3.0:1787828444.362332:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828444.362337:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828444.362344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828444.362357:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476829696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828444.362367:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828444.362372:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828444.362375:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828444.362383:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2811 00000100:00080000:3.0:1787828444.362389:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828444.362394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9a78a6775680 x1874671476829312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828444.362402:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828444.362421:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828444.362431:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476829312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828444.362445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1787828444.362448:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476829824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828444.362453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828444.362468:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476829312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828444.362508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828444.362554:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828444.362575:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9a78a6776d00 x1874671476829440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787828444.362580:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828444.362599:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476829440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828444.362610:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828444.362614:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828444.362617:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828444.362624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828444.362641:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828444.362650:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9a78a67761c0 x1874671476829568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828444.362674:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476829568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828444.362682:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828444.362685:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828444.362689:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828444.362695:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828444.362723:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828444.362733:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9a78a6774780 x1874671476829696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828444.362743:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476829696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828444.362761:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828444.362779:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828444.362782:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828444.362792:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828444.362807:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828444.362817:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828444/real 1787828444] req@ffff9a78a67770c0 x1874671476829824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828444.362872:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476829824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828444.362883:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828444.362889:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828444.362894:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476829824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828445.385965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828446.410043:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828447.434018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828448.458087:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828449.481954:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828449.481964:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828449.481969:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828449.481980:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2816 00000100:00080000:1.0:1787828449.481992:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828449.481999:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828449.482001:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828449.482049:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476829952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828449.482083:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828449.482087:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828449.482089:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828449.482092:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2816 00000100:00080000:1.0:1787828449.482095:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828449.482098:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828449.482127:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828449.482163:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476830080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828449.482179:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828449.482184:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828449.482188:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828449.482195:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476829952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828449.482199:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2816 00000100:00080000:1.0:1787828449.482210:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828449.482219:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828449.482227:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828449.482247:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476830208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828449.482273:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828449.482277:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828449.482280:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787828449.482284:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828449.482291:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2816 00000100:00080000:1.0:1787828449.482300:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828449.482307:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828449.482314:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828449.482334:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476830336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828449.482344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828449/real 1787828449] req@ffff9a7890b1c000 x1874671476829952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828449.482354:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828449.482364:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828449.482368:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828449.482373:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476829952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828449.482379:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2816 00000100:00080000:1.0:1787828449.482390:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828449.482395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828449.482399:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787828449.482404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:1.0:1787828449.482411:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828449.482412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476829952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828449.482426:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828449.482439:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476830464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828449.482453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828449.482475:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828449/real 1787828449] req@ffff9a7890b1fc00 x1874671476830080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828449.482494:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476830080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828449.482506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828449.482526:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828449.482531:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828449.482580:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828449.482608:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828449.482623:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828449/real 1787828449] req@ffff9a7890b1f840 x1874671476830208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828449.482637:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476830208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828449.482647:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828449.482650:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828449.482654:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828449.482660:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828449.482677:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828449.482701:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828449/real 1787828449] req@ffff9a7890b1f480 x1874671476830336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828449.482715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476830336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828449.482726:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828449.482729:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828449.482733:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828449.482739:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828449.482759:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828449.482786:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828449/real 1787828449] req@ffff9a7890b1f0c0 x1874671476830464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828449.482796:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476830464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828449.482805:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828449.482808:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828449.482810:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830464:192.168.203.140@tcp:8:kworker.0 00000008:00020000:1.0:1787828450.506014:0:47597:0:(osc_cache.c:999:osc_extent_wait()) extent ffff9a7886b83b40@{[0 -> 0/1023], [3|0|+|rpc|wiuY|ffff9a78a67f2000], [28672|1|+|-|ffff9a7883550c00|1024|ffff9a7884d8cf80]} lustre-OST0000-osc-ffff9a78a6512000: wait ext to 0 timedout, recovery in progress? 00000008:00020000:2.0:1787828450.506049:0:47598:0:(osc_cache.c:999:osc_extent_wait()) extent ffff9a7887a576c0@{[0 -> 0/1023], [3|0|+|rpc|wiuY|ffff9a788626c8c0], [28672|1|+|-|ffff9a78869e0c00|1024|ffff9a78997f8000]} lustre-OST0001-osc-ffff9a78a6512000: wait ext to 0 timedout, recovery in progress? 00000008:00020000:2.0:1787828450.506081:0:47598:0:(osc_cache.c:999:osc_extent_wait()) ### extent: 00000000f2fcd4bd ns: lustre-OST0001-osc-ffff9a78a6512000 lock: 000000000863a79e/0xebdcd48a4298e0fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x7b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800029400020000 nid: local remote: 0x4059aa0aef8da261 expref: -99 pid: 48370 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787828450.506171:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00020000:1.0:1787828450.531335:0:47597:0:(osc_cache.c:999:osc_extent_wait()) ### extent: 000000002d35b46b ns: lustre-OST0000-osc-ffff9a78a6512000 lock: 000000009f81abd8/0xebdcd48a4298e8d3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x90:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800029400020000 nid: local remote: 0x4059aa0aef8dbca8 expref: -99 pid: 48385 timeout: 0 lvb_type: 1 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787828451.530025:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828452.554055:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828453.578042:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828454.602034:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828454.602035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828454.602048:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828454.602055:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828454.602069:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2822 00000100:00080000:1.0:1787828454.602111:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828454.602121:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828454.602171:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c3c0 x1874671476830592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828454.602210:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828454.602215:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828454.602219:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828454.602226:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2822 00000100:00080000:1.0:1787828454.602231:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828454.602237:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828454.602250:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828454.602260:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476830592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828454.602272:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476830720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828454.602291:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828454.602296:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828454.602299:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828454.602304:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2822 00000100:00080000:1.0:1787828454.602307:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828454.602314:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828454.602323:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828454.602338:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476830848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828454.602359:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828454.602366:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828454.602371:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828454.602378:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2822 00000100:00080000:1.0:1787828454.602386:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828454.602394:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828454.602405:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828454.602421:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828454.602437:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476830976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828454.602457:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830848:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787828454.602467:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828454.602476:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828454.602481:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828454.602495:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830976:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787828454.602495:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2822 00000100:00080000:1.0:1787828454.602510:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828454.602520:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828454.602530:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828454.602548:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476831104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828454.602591:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828455.626100:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828456.650157:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828457.674295:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00010000:00010000:1.0:1787828457.674354:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828457.674358:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828457.674386:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828457.674452:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828457.674474:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828457.674494:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828457.674510:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787828457.674543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828454/real 1787828457] req@ffff9a7890b1e940 x1874671476831104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828457.674592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476831104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828457.674616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828457.674625:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828457.674634:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831104:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828457.674650:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828454/real 1787828457] req@ffff9a7890b1de00 x1874671476830976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828457.674676:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476830976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828457.674709:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828457.674717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828457.674724:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828457.674739:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828454/real 1787828457] req@ffff9a7890b1d680 x1874671476830720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828457.674767:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476830720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828457.674785:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828457.674792:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828457.674799:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828457.674817:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828454/real 1787828457] req@ffff9a7890b1e580 x1874671476830848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828457.674901:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476830848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828457.674918:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828457.674925:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828457.674932:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476830848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828457.674945:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828454/real 1787828457] req@ffff9a7890b1c3c0 x1874671476830592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828457.674972:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c3c0 x1874671476830592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828457.674988:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828457.674994:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828457.675001:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476830592:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787828458.698184:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828459.721985:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828459.721998:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828459.722003:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828459.722017:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2827 00000100:00080000:2.0:1787828459.722032:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828459.722069:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828459.722118:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476831232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828459.722158:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828459.722177:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:0.0:1787828459.722178:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828459.722182:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828459.722192:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2827 00000100:00080000:2.0:1787828459.722199:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828459.722205:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828459.722220:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828459.722240:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476831232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828459.722241:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476831360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828459.722278:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828459.722286:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828459.722290:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828459.722298:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2827 00000100:00080000:2.0:1787828459.722306:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828459.722314:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828459.722324:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787828459.722377:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828459.722380:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476831488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828459.722406:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828459.722414:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828459.722417:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828459.722426:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2827 00000100:00080000:2.0:1787828459.722435:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828459.722444:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828459.722456:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828459.722464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828459/real 1787828459] req@ffff9a78841dbc00 x1874671476831232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828459.722490:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476831616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828459.722512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476831232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828459.722520:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828459.722541:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828459.722548:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828459.722559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828459.722563:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2827 00000100:00080000:2.0:1787828459.722578:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828459.722579:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787828459.722598:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828459.722599:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476831232:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828459.722621:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828459.722654:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476831744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828459.722660:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828459.722731:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828459.722764:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828459/real 1787828459] req@ffff9a78841db0c0 x1874671476831360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828459.722791:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476831360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828459.722809:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828459.722817:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828459.722824:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828459.722875:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828459.722919:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828459.722945:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828459/real 1787828459] req@ffff9a78841da1c0 x1874671476831488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828459.722972:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476831488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828459.722989:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828459.722996:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828459.723002:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828459.723016:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828459.723098:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828459.723123:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828459/real 1787828459] req@ffff9a78841d9a40 x1874671476831616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828459.723147:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476831616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828459.723164:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828459.723170:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828459.723175:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828459.723208:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828459.723245:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828459.723269:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828459/real 1787828459] req@ffff9a78841d8780 x1874671476831744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828459.723292:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476831744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828459.723324:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828459.723331:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828459.723337:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476831744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828460.746350:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828461.769966:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828462.793988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828463.818065:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828464.841945:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828464.841958:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828464.841963:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828464.841978:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2832 00000100:00080000:2.0:1787828464.841992:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828464.842044:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828464.842060:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828464.842078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476831872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828464.842100:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828464.842104:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828464.842106:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828464.842112:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2832 00000100:00080000:2.0:1787828464.842116:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828464.842120:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828464.842127:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476831872:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828464.842130:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828464.842148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476832000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828464.842163:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828464.842166:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828464.842167:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828464.842171:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2832 00000100:00080000:2.0:1787828464.842174:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828464.842177:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828464.842181:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787828464.842182:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828464.842196:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476832128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828464.842203:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828464.842206:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828464.842207:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828464.842210:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2832 00000100:00080000:2.0:1787828464.842212:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828464.842214:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828464.842217:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828464.842224:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476832256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828464.842230:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828464.842233:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828464.842235:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828464.842239:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828464/real 1787828464] req@ffff9a78841da580 x1874671476831872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828464.842239:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2832 00000100:00080000:2.0:1787828464.842263:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828464.842270:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828464.842278:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828464.842290:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476832384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828464.842299:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476831872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828464.842330:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828464.842336:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828464.842341:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476831872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828464.842362:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828464.842398:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828464.842410:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828464/real 1787828464] req@ffff9a78841d8000 x1874671476832000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828464.842419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476832000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828464.842426:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828464.842429:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828464.842431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828464.842436:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828464.842450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828464.842457:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828464/real 1787828464] req@ffff9a78841d83c0 x1874671476832128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828464.842465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476832128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828464.842475:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828464.842478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828464.842480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828464.842484:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828464.842495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828464.842502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828464/real 1787828464] req@ffff9a78841db480 x1874671476832256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828464.842510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476832256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828464.842517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828464.842519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828464.842521:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828464.842525:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828464.842535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828464.842541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828464/real 1787828464] req@ffff9a78841d9e00 x1874671476832384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828464.842549:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476832384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828464.842556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828464.842558:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828464.842561:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828465.866111:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828466.889972:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828467.914055:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828468.938107:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828469.961927:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828469.961935:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828469.961939:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828469.961948:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2837 00000100:00080000:0.0:1787828469.961957:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828469.961980:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828469.961981:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828469.962026:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476832512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828469.962047:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828469.962050:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828469.962051:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828469.962054:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2837 00000100:00080000:0.0:1787828469.962056:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828469.962071:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828469.962079:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828469.962090:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476832512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828469.962091:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476832640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828469.962104:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828469.962106:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828469.962107:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828469.962109:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2837 00000100:00080000:0.0:1787828469.962112:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828469.962114:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828469.962119:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828469.962132:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476832768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828469.962142:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828469.962146:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828469.962148:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828469.962152:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2837 00000100:00080000:0.0:1787828469.962157:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787828469.962162:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828469.962166:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828469.962182:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828469.962195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476832896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828469.962202:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828469.962205:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828469.962206:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828469.962210:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2837 00000100:00080000:0.0:1787828469.962214:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828469.962219:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828469.962219:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828469/real 1787828469] req@ffff9a7890a3da40 x1874671476832512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828469.962224:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828469.962235:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476833024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828469.962247:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476832512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828469.962261:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828469.962267:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828469.962272:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476832512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828469.962292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828469.962310:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828469.962320:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828469/real 1787828469] req@ffff9a7890a3de00 x1874671476832640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828469.962328:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476832640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828469.962334:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828469.962336:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828469.962338:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828469.962342:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828469.962353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828469.962359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828469/real 1787828469] req@ffff9a7890a3f480 x1874671476832768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828469.962366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476832768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828469.962371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828469.962373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828469.962375:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828469.962378:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828469.962387:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828469.962393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828469/real 1787828469] req@ffff9a7890a3c3c0 x1874671476832896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828469.962400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476832896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828469.962407:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828469.962408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828469.962410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476832896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828469.962413:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828469.962422:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828469.962428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828469/real 1787828469] req@ffff9a7890a3c000 x1874671476833024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828469.962439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476833024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828494 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828469.962446:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828469.962447:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828469.962449:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828470.986040:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828472.010150:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828473.034053:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828474.058027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828475.082051:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828475.082062:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828475.082067:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828475.082071:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828475.082079:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2842 00000100:00080000:0.0:1787828475.082091:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828475.082110:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828475.082155:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476833152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828475.082192:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828475.082196:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828475.082199:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828475.082203:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2842 00000100:00080000:0.0:1787828475.082207:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828475.082212:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828475.082222:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828475.082246:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476833280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828475.082259:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828475.082263:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828475.082266:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828475.082271:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2842 00000100:00080000:0.0:1787828475.082274:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828475.082278:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476833152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828475.082283:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828475.082290:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828475.082314:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476833408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828475.082327:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828475.082332:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828475.082334:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828475.082340:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2842 00000100:00080000:0.0:1787828475.082344:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787828475.082350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828475.082351:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828475.082360:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828475.082373:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476833536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828475.082393:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828475.082398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828475] req@ffff9a7890a3fc00 x1874671476833152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828475.082400:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828475.082404:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828475.082416:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2842 00000100:00080000:0.0:1787828475.082421:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828475.082427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476833152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828475.082428:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828475.082437:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:1.0:1787828475.082443:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828475.082448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828475.082456:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476833152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828475.082457:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476833664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828475.082467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828475.082492:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828475.082502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828475] req@ffff9a7890a3c000 x1874671476833280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828475.082517:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476833280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828475.082523:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828475.082525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828475.082527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828475.082546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828475.082559:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828475.082570:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828475] req@ffff9a7890a3c3c0 x1874671476833408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828475.082583:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476833408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828475.082593:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828475.082596:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828475.082600:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828475.082608:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828475.082630:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828475.082643:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828475] req@ffff9a7890a3f480 x1874671476833536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828475.082652:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476833536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828475.082664:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828475.082666:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828475.082669:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828475.082673:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828475.082687:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828475.082694:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828475/real 1787828475] req@ffff9a7890a3de00 x1874671476833664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828475.082702:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476833664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828475.082708:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828475.082710:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828475.082713:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828476.106060:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828477.129974:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828478.154074:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828479.178026:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828480.201981:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828480.201990:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828480.201995:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828480.202004:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2847 00000100:00080000:3.0:1787828480.202014:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828480.202019:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828480.202052:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476833792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828480.202076:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828480.202079:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828480.202081:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828480.202084:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2847 00000100:00080000:3.0:1787828480.202087:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828480.202090:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828480.202098:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828480.202106:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476833920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828480.202114:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828480.202116:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828480.202118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828480.202120:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2847 00000100:00080000:3.0:1787828480.202122:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828480.202124:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828480.202127:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828480.202135:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476834048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828480.202143:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828480.202146:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828480.202147:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828480.202150:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2847 00000100:00080000:3.0:1787828480.202152:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828480.202154:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828480.202158:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828480.202164:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476833792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828480.202170:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476834176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828480.202184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828480.202187:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828480.202188:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828480.202190:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2847 00000100:00080000:3.0:1787828480.202193:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828480.202196:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828480.202200:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828480.202209:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476834304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828480.202225:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828480.202272:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9a78a6776940 x1874671476833792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828480.202287:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476833792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828480.202301:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828480.202304:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828480.202308:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476833792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828480.202314:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833920:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787828480.202328:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:1.0:1787828480.202333:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828480.202345:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9a78a6776580 x1874671476833920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828480.202354:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476833920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828480.202360:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828480.202362:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828480.202365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476833920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828480.202368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828480.202379:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828480.202385:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9a78a6774b40 x1874671476834048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828480.202393:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476834048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828480.202399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828480.202401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828480.202404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828480.202427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828480.202438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828480.202445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9a78a67752c0 x1874671476834176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828480.202453:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476834176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828480.202459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828480.202461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828480.202463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828480.202467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828480.202477:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828480.202483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828480/real 1787828480] req@ffff9a78a6775a40 x1874671476834304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828480.202491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476834304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828480.202497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828480.202499:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828480.202501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828481.226096:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828482.250001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828483.273996:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828484.297961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828485.321907:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828485.321918:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828485.321922:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828485.321932:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2852 00000100:00080000:3.0:1787828485.321943:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828485.321951:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828485.321986:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476834432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828485.322015:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828485.322019:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828485.322021:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828485.322024:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2852 00000100:00080000:3.0:1787828485.322027:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828485.322030:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828485.322040:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828485.322051:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476834560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828485.322062:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828485.322082:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828485.322084:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828485.322087:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2852 00000100:00080000:3.0:1787828485.322090:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828485.322092:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828485.322096:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828485.322110:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476834688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828485.322111:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476834432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828485.322132:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828485.322137:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828485.322140:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828485.322145:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2852 00000100:00080000:3.0:1787828485.322150:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828485.322155:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828485.322162:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828485.322177:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476834816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828485.322206:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:1.0:1787828485.322212:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828485.322213:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828485.322218:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828485.322223:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2852 00000100:00080000:3.0:1787828485.322228:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828485.322232:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828485.322238:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828485.322250:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476834944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828485.322298:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9a78a6774000 x1874671476834432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828485.322323:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476834432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828485.322353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:0.0:1787828485.322357:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828485.322361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828485.322369:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476834432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828485.322380:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828485.322406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828485.322421:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9a78a6777480 x1874671476834560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828485.322437:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476834560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828485.322447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828485.322467:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828485.322471:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828485.322507:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828485.322531:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828485.322543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9a78a6775a40 x1874671476834688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828485.322566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476834688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828485.322575:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828485.322580:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828485.322584:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828485.322591:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828485.322613:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828485.322627:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9a78a67752c0 x1874671476834816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828485.322641:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476834816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828485.322652:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828485.322657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828485.322661:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828485.322668:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828485.322689:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828485.322702:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828485/real 1787828485] req@ffff9a78a6774b40 x1874671476834944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828485.322716:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476834944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828485.322734:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828485.322738:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828485.322742:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476834944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828486.345967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828487.369968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828488.393992:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828489.417937:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828490.441969:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828490.441980:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828490.441986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828490.441997:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2857 00000100:00080000:1.0:1787828490.442009:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828490.442017:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828490.442025:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828490.442059:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476835072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828490.442129:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828490.442134:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828490.442137:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828490.442142:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2857 00000100:00080000:1.0:1787828490.442148:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828490.442154:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828490.442167:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828490.442186:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476835200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828490.442202:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828490.442208:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787828490.442214:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476835072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828490.442214:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828490.442225:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2857 00000100:00080000:1.0:1787828490.442237:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828490.442241:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828490.442247:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828490.442264:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476835328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828490.442277:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828490.442282:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828490.442285:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828490.442289:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2857 00000100:00080000:1.0:1787828490.442323:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828490.442326:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828490.442331:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828490.442335:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828490.442347:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476835456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828490.442355:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828490.442358:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828490.442359:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828490.442362:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2857 00000100:00080000:1.0:1787828490.442365:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828490.442368:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828490.442371:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828490.442382:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476835584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828490.442387:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff9a7890b1e580 x1874671476835072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828490.442432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476835072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828490.442446:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828490.442452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828490.442457:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476835072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828490.442490:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828490.442535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828490.442561:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff9a7890b1d680 x1874671476835200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828490.442625:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476835200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828490.442632:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828490.442634:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828490.442637:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828490.442642:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828490.442664:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828490.442678:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff9a7890b1de00 x1874671476835328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828490.442693:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476835328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828490.442703:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828490.442707:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828490.442711:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828490.442718:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828490.442741:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828490.442755:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff9a7890b1e940 x1874671476835456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828490.442770:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476835456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828490.442781:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828490.442785:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828490.442789:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828490.442796:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828490.442817:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828490.442881:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828490/real 1787828490] req@ffff9a7890b1da40 x1874671476835584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828490.442894:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476835584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828490.442909:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828490.442912:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828490.442915:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828491.466092:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828492.490033:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828493.513954:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828494.538010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828495.561935:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828495.561945:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828495.561950:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828495.561961:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2863 00000100:00080000:1.0:1787828495.561972:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828495.561979:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828495.562019:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476835712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828495.562044:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828495.562048:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828495.562054:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828495.562056:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828495.562066:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2863 00000100:00080000:1.0:1787828495.562069:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828495.562073:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828495.562082:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828495.562094:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476835840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828495.562105:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828495.562109:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828495.562111:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828495.562117:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2863 00000100:00100000:0.0:1787828495.562119:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476835712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828495.562120:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828495.562128:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828495.562142:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828495.562155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476835968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828495.562164:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828495.562168:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828495.562170:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828495.562173:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2863 00000100:00080000:1.0:1787828495.562180:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828495.562183:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828495.562189:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828495.562194:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828495.562205:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476836096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828495.562215:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828495.562219:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828495.562220:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828495.562224:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2863 00000100:00080000:1.0:1787828495.562229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828495.562235:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828495.562242:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828495.562247:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828495/real 1787828495] req@ffff9a7890b1da40 x1874671476835712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828495.562262:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476836224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828495.562270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476835712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828495.562297:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828495.562302:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828495.562306:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476835712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828495.562331:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828495.562350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828495.562361:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828495/real 1787828495] req@ffff9a7890b1e940 x1874671476835840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828495.562372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476835840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828495.562379:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828495.562383:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828495.562386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828495.562392:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828495.562404:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828495.562412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828495/real 1787828495] req@ffff9a7890b1de00 x1874671476835968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828495.562422:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476835968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828495.562429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828495.562432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828495.562434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476835968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828495.562439:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828495.562451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828495.562459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828495/real 1787828495] req@ffff9a7890b1d680 x1874671476836096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828495.562481:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476836096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828495.562489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828495.562492:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828495.562495:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828495.562500:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828495.562515:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828495.562524:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828495/real 1787828495] req@ffff9a7890b1e580 x1874671476836224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828495.562534:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476836224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828495.562541:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828495.562544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828495.562546:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828496.586045:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828497.610145:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828498.634001:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828499.658159:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828500.681948:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828500.681961:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828500.681979:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828500.681993:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2868 00000100:00080000:1.0:1787828500.682007:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828500.682018:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828500.682060:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828500.682068:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476836352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828500.682105:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828500.682110:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828500.682113:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828500.682117:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2868 00000100:00080000:1.0:1787828500.682121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828500.682127:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828500.682139:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828500.682162:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476836352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828500.682167:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476836480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828500.682197:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828500.682204:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828500.682209:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828500.682218:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2868 00000100:00080000:1.0:1787828500.682239:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787828500.682253:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828500.682254:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828500.682281:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828500.682308:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476836608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828500.682323:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828500.682330:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1787828500.682337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828500/real 1787828500] req@ffff9a7890b1f0c0 x1874671476836352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828500.682337:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828500.682349:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2868 00000100:00080000:1.0:1787828500.682361:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828500.682374:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828500.682385:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828500.682404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476836352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828500.682406:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476836736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828500.682427:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828500.682431:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828500.682433:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828500.682440:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2868 00000100:00080000:1.0:1787828500.682454:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828500.682470:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787828500.682482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787828500.682483:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787828500.682497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828500.682505:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476836352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828500.682519:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476836864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828500.682545:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828500.682587:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828500.682606:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828500/real 1787828500] req@ffff9a7890b1f480 x1874671476836480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828500.682631:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476836480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828500.682642:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828500.682646:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828500.682650:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828500.682657:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828500.682676:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828500.682687:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828500/real 1787828500] req@ffff9a7890b1f840 x1874671476836608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828500.682701:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476836608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828500.682711:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828500.682714:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828500.682718:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828500.682724:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828500.682742:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828500.682785:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828500/real 1787828500] req@ffff9a7890b1fc00 x1874671476836736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828500.682803:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476836736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828500.682823:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828500.682878:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828500.682884:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828500.682908:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828500.682937:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828500.682952:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828500/real 1787828500] req@ffff9a7890b1c000 x1874671476836864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828500.682967:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476836864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828500.682978:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828500.682982:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828500.682986:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476836864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828501.705966:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828502.729952:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828503.753932:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828504.778004:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828505.801948:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828505.801961:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828505.801966:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828505.801979:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2873 00000100:00080000:3.0:1787828505.801992:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828505.802001:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828505.802046:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476836992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828505.802083:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828505.802088:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828505.802090:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828505.802095:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2873 00000100:00080000:3.0:1787828505.802099:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828505.802106:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828505.802118:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828505.802144:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476837120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828505.802151:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476836992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828505.802198:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828505.802206:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828505.802212:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828505.802223:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2873 00000100:00080000:3.0:1787828505.802246:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828505.802254:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828505.802262:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787828505.802270:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828505.802298:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476837248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828505.802314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828505.802319:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828505.802323:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828505.802333:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2873 00000100:00100000:0.0:1787828505.802334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828505/real 1787828505] req@ffff9a78a6776580 x1874671476836992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828505.802338:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828505.802352:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828505.802366:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828505.802380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476836992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828505.802388:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476837376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828505.802402:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828505.802420:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828505.802423:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828505.802425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828505.802431:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2873 00000100:00080000:0.0:1787828505.802440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787828505.802443:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828505.802455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476836992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828505.802457:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828505.802474:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828505.802490:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476837504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828505.802506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828505.802545:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828505.802580:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828505/real 1787828505] req@ffff9a78a6776940 x1874671476837120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828505.802601:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476837120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828505.802614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828505.802621:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828505.802634:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837120:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787828505.802637:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828505.802647:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828505.802675:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828505.802688:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828505/real 1787828505] req@ffff9a78a67770c0 x1874671476837248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828505.802703:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476837248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828505.802712:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828505.802741:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828505.802746:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828505.802753:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828505.802776:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828505.802790:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828505/real 1787828505] req@ffff9a78a6774780 x1874671476837376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828505.802804:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476837376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828505.802817:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828505.802820:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828505.802824:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828505.802886:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828505.802912:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828505.802926:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828505/real 1787828505] req@ffff9a78a67761c0 x1874671476837504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828505.802941:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476837504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828505.802953:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828505.802957:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828505.802961:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828506.826153:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828507.849986:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828508.874018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828509.898061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828510.921948:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828510.921960:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828510.921966:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828510.921978:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2878 00000100:00080000:3.0:1787828510.922002:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828510.922010:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828510.922070:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476837632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828510.922096:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828510.922099:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828510.922101:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828510.922105:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2878 00000100:00080000:3.0:1787828510.922108:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828510.922111:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828510.922138:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828510.922161:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476837760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828510.922172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828510.922180:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828510.922182:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828510.922188:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2878 00000100:00080000:3.0:1787828510.922192:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828510.922197:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828510.922203:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828510.922216:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476837888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828510.922243:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828510.922244:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476837632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828510.922249:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828510.922254:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828510.922259:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2878 00000100:00080000:3.0:1787828510.922262:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828510.922265:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828510.922270:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828510.922281:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476838016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828510.922288:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828510.922291:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828510.922293:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828510.922296:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2878 00000100:00080000:3.0:1787828510.922319:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828510.922323:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787828510.922329:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787828510.922330:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828510.922343:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476838144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828510.922384:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828510/real 1787828510] req@ffff9a78a6776d00 x1874671476837632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828510.922413:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476837632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828510.922429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828510.922435:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00010000:00010000:2.0:1787828510.922441:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828510.922442:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476837632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828510.922458:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828510.922488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828510.922505:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828510/real 1787828510] req@ffff9a78a6775680 x1874671476837760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828510.922517:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476837760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828510.922524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828510.922545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828510.922549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828510.922579:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828510.922611:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828510.922623:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828510/real 1787828510] req@ffff9a78a6775e00 x1874671476837888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828510.922637:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476837888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828510.922654:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828510.922657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828510.922661:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476837888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828510.922667:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828510.922686:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828510.922743:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828510/real 1787828510] req@ffff9a78a6777840 x1874671476838016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828510.922759:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476838016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828510.922772:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828510.922776:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828510.922779:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828510.922784:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828510.922803:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828510.922812:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828510/real 1787828510] req@ffff9a78a6774f00 x1874671476838144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828510.922822:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476838144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828510.922854:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828510.922857:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828510.922860:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828511.946113:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828512.970062:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828513.994091:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828515.018099:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828516.041955:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828516.041980:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828516.041986:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828516.041999:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2883 00000100:00080000:3.0:1787828516.042012:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.042021:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828516.042068:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476838272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828516.042104:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828516.042109:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828516.042111:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828516.042116:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2883 00000100:00080000:3.0:1787828516.042120:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.042125:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828516.042137:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828516.042154:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476838400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828516.042177:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828516.042180:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476838272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828516.042185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828516.042197:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828516.042207:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2883 00000100:00080000:3.0:1787828516.042216:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.042224:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828516.042235:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828516.042264:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476838528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828516.042271:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828516.042295:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828516.042300:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828516.042303:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828516.042312:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2883 00000100:00080000:3.0:1787828516.042317:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.042323:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828516.042333:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828516.042356:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff9a78a67743c0 x1874671476838272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828516.042359:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476838656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828516.042405:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476838272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828516.042421:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828516.042426:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828516.042430:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828516.042437:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787828516.042440:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828516.042451:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2883 00000100:00100000:0.0:1787828516.042464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476838272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828516.042467:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828516.042483:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828516.042494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828516.042506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828516.042522:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476838784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828516.042556:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828516.042581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff9a78a6777c00 x1874671476838400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828516.042601:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476838400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828516.042613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828516.042618:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828516.042623:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828516.042636:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828516.042713:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:1.0:1787828516.042719:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828516.042740:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff9a78a6774f00 x1874671476838528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828516.042758:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476838528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828516.042769:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828516.042774:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828516.042778:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828516.042786:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828516.042806:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828516.042818:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff9a78a6777840 x1874671476838656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828516.042867:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476838656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828516.042880:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828516.042884:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828516.042888:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828516.042912:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828516.042935:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828516.042949:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828516/real 1787828516] req@ffff9a78a6775e00 x1874671476838784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828516.042994:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476838784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828516.043013:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828516.043020:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828516.043028:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476838784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828517.066340:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828518.090121:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828519.114144:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828520.138015:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828521.161947:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828521.161955:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828521.161960:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828521.161970:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2888 00010000:00010000:2.0:1787828521.161975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828521.161979:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828521.161986:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828521.162019:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476838912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828521.162040:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828521.162043:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828521.162044:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828521.162047:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2888 00000100:00080000:1.0:1787828521.162050:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828521.162052:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828521.162060:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828521.162069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476839040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828521.162076:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828521.162078:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828521.162079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828521.162081:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2888 00000100:00080000:1.0:1787828521.162083:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828521.162085:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828521.162088:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828521.162095:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476839168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828521.162107:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828521.162110:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828521.162111:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828521.162116:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476838912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828521.162117:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2888 00000100:00080000:1.0:1787828521.162123:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828521.162126:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828521.162129:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828521.162138:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476839296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828521.162144:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828521.162147:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828521.162148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828521.162151:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2888 00000100:00080000:1.0:1787828521.162168:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828521.162173:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828521.162178:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828521.162192:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476839424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828521.162214:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828521.162246:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828521.162259:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828521.162272:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828522.186174:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828523.210133:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828524.234513:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00010000:00010000:2.0:1787828524.234646:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828524.234649:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828524.234709:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828524.234808:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828524.234825:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828524.234920:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787828524.234935:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828524] req@ffff9a7890b1f840 x1874671476839168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:1.0:1787828524.234964:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787828524.235019:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476839168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828524.235045:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828524.235054:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828524.235063:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828524.235078:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828524] req@ffff9a7890b1f480 x1874671476839296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828524.235119:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476839296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828524.235138:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828524.235144:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828524.235151:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828524.235173:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828524] req@ffff9a7890b1f0c0 x1874671476839424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828524.235196:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476839424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828524.235215:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828524.235221:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828524.235227:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839424:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828524.235240:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828524] req@ffff9a7890b1c000 x1874671476838912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828524.235263:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476838912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828524.235278:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828524.235285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828524.235291:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476838912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828524.235302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828521/real 1787828524] req@ffff9a7890b1fc00 x1874671476839040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828524.235323:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476839040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828524.235337:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828524.235342:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828524.235347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839040:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787828525.258111:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828526.281998:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828526.282013:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828526.282020:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828526.282037:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2893 00000100:00080000:0.0:1787828526.282054:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828526.282066:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828526.282139:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476839552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828526.282181:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828526.282186:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828526.282188:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828526.282193:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2893 00000100:00080000:0.0:1787828526.282199:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787828526.282203:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828526.282205:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828526.282223:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828526.282240:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476839680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828526.282264:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828526.282270:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828526.282274:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828526.282281:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2893 00000100:00080000:0.0:1787828526.282287:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828526.282296:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828526.282301:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476839552:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787828526.282306:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828526.282331:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476839808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828526.282347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828526.282353:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828526.282356:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828526.282363:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2893 00000100:00080000:0.0:1787828526.282369:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828526.282376:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828526.282384:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828526.282403:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476839936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828526.282419:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828526.282424:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828526.282428:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828526.282434:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2893 00000100:00080000:0.0:1787828526.282442:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787828526.282450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828526.282455:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828526.282466:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828526.282487:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476840064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828526.282543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9a7890a3da40 x1874671476839552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828526.282594:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476839552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828526.282614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828526.282621:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828526.282628:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476839552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828526.282641:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828526.282679:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828526.282701:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9a7890a3de00 x1874671476839680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828526.282721:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476839680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828526.282734:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828526.282740:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828526.282746:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828526.282789:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828526.282822:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828526.282882:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9a7890a3f480 x1874671476839808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828526.282902:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476839808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828526.282914:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828526.282919:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828526.282924:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828526.282933:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828526.282967:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828526.282985:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9a7890a3c3c0 x1874671476839936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828526.283002:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476839936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828526.283018:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828526.283023:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828526.283028:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476839936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828526.283037:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828526.283069:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828526.283085:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828526/real 1787828526] req@ffff9a7890a3c000 x1874671476840064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828526.283103:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476840064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828526.283117:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828526.283121:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828526.283126:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828527.306232:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828528.330128:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828529.354078:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828530.378066:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828531.401913:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828531.401923:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828531.401928:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828531.401937:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2898 00000100:00080000:0.0:1787828531.401949:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828531.401958:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828531.401975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828531.402014:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476840192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828531.402040:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828531.402044:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828531.402046:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828531.402049:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2898 00000100:00080000:0.0:1787828531.402053:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828531.402057:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828531.402067:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828531.402081:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476840192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828531.402084:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476840320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828531.402097:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828531.402103:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828531.402105:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828531.402108:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2898 00000100:00080000:0.0:1787828531.402111:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828531.402115:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828531.402119:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828531.402129:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476840448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828531.402137:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828531.402140:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828531.402141:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828531.402146:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2898 00000800:00000100:1.0:1787828531.402152:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828531.402154:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828531.402161:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828531.402166:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828531.402177:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476840576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828531.402184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828531.402187:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828531.402189:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828531.402192:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2898 00000100:00080000:0.0:1787828531.402195:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828531.402199:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828531.402204:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828531.402220:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476840704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828531.402220:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9a7890a3fc00 x1874671476840192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828531.402272:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476840192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828531.402287:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828531.402293:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828531.402298:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476840192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828531.402325:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828531.402348:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828531.402360:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9a7890a3c000 x1874671476840320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828531.402371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476840320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828531.402378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828531.402381:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828531.402384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828531.402389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787828531.402402:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828531.402409:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9a7890a3c3c0 x1874671476840448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828531.402418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476840448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828531.402424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828531.402426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828531.402429:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828531.402433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828531.402446:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828531.402453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9a7890a3f480 x1874671476840576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828531.402462:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476840576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828531.402469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828531.402471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828531.402474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787828531.402479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787828531.402490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828531.402497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828531/real 1787828531] req@ffff9a7890a3de00 x1874671476840704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828531.402505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476840704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828531.402513:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828531.402515:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787828531.402517:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828532.425986:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828533.449996:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828534.474030:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828535.498153:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828536.521929:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828536.521938:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828536.521943:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828536.521951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828536.521954:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2904 00000100:00080000:1.0:1787828536.521967:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828536.521974:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828536.522029:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476840832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828536.522059:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828536.522063:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828536.522065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828536.522070:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2904 00000100:00080000:1.0:1787828536.522074:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828536.522079:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828536.522089:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828536.522103:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476840832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828536.522105:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476840960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828536.522120:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828536.522124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828536.522126:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828536.522130:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2904 00000100:00080000:1.0:1787828536.522133:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828536.522137:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828536.522142:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828536.522155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476841088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828536.522165:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828536.522169:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828536.522172:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787828536.522175:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828536.522179:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2904 00000100:00080000:1.0:1787828536.522184:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828536.522189:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828536.522195:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828536.522209:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476841216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828536.522222:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828536/real 1787828536] req@ffff9a7890b1e580 x1874671476840832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828536.522230:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828536.522236:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828536.522238:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828536.522242:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476840832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828536.522247:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2904 00000100:00080000:0.0:1787828536.522255:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828536.522256:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828536.522261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828536.522266:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476840832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828536.522266:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828536.522277:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828536.522295:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828536.522295:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476841344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828536.522317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828536.522328:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828536/real 1787828536] req@ffff9a7890b1d680 x1874671476840960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828536.522337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476840960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828536.522343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828536.522345:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828536.522347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476840960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828536.522353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828536.522363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828536.522369:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828536/real 1787828536] req@ffff9a7890b1de00 x1874671476841088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828536.522376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476841088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828536.522381:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828536.522383:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828536.522385:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828536.522389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828536.522399:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828536.522404:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828536/real 1787828536] req@ffff9a7890b1e940 x1874671476841216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828536.522411:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476841216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828536.522417:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828536.522419:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828536.522421:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828536.522431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828536.522441:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828536.522447:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828536/real 1787828536] req@ffff9a7890b1da40 x1874671476841344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828536.522454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476841344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828536.522460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828536.522462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828536.522464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828537.545975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828538.570112:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828539.593978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828540.618161:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828541.642037:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828541.642049:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828541.642056:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828541.642070:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2909 00000100:00080000:2.0:1787828541.642094:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828541.642104:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828541.642148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476841472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828541.642179:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828541.642191:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828541.642200:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828541.642205:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828541.642217:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2909 00000100:00080000:2.0:1787828541.642224:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828541.642232:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828541.642246:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828541.642252:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476841472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828541.642274:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476841600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828541.642291:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828541.642296:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828541.642298:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828541.642313:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2909 00000100:00080000:2.0:1787828541.642335:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828541.642341:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828541.642348:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828541.642366:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476841728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828541.642378:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828541.642382:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828541.642384:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828541.642391:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2909 00000100:00080000:2.0:1787828541.642396:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828541.642400:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828541.642406:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828541.642411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828541.642428:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476841856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828541.642451:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828541.642456:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828541.642459:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828541.642463:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2909 00000100:00080000:2.0:1787828541.642491:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828541.642497:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828541.642516:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828541.642544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828541/real 1787828541] req@ffff9a78841d8b40 x1874671476841472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828541.642552:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476841984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828541.642625:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476841472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828541.642641:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828541.642646:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828541.642652:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476841472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828541.642685:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828541.642711:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828541.642740:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828541/real 1787828541] req@ffff9a78841db840 x1874671476841600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828541.642757:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476841600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828541.642768:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828541.642773:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828541.642777:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828541.642785:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828541.642807:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828541.642821:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828541/real 1787828541] req@ffff9a78841d92c0 x1874671476841728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828541.642835:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476841728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828541.642845:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828541.642849:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828541.642852:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828541.642894:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828541.642922:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828541.642937:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828541/real 1787828541] req@ffff9a78841d9680 x1874671476841856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828541.642952:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476841856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828541.642964:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828541.642969:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828541.642972:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828541.642981:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828541.643000:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828541.643011:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828541/real 1787828541] req@ffff9a78841da940 x1874671476841984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828541.643025:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476841984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828541.643038:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828541.643042:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828541.643046:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476841984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828542.666109:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828543.690150:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828544.714089:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828545.738048:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828546.761972:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828546.761988:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828546.761996:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828546.762013:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2914 00000100:00080000:2.0:1787828546.762033:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828546.762044:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828546.762114:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476842112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828546.762130:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828546.762157:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828546.762165:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828546.762168:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828546.762173:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2914 00000100:00080000:2.0:1787828546.762179:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828546.762186:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828546.762202:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828546.762220:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476842240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828546.762240:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476842112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828546.762260:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828546.762266:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828546.762269:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828546.762276:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2914 00000100:00080000:2.0:1787828546.762280:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828546.762286:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828546.762292:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828546.762309:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476842368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828546.762322:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828546.762327:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828546.762330:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828546.762335:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2914 00000100:00080000:2.0:1787828546.762360:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828546.762365:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828546.762372:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828546.762381:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828546.762391:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476842496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828546.762416:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828546.762421:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828546.762423:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828546.762448:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2914 00000100:00080000:2.0:1787828546.762453:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828546.762458:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828546.762464:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828546.762478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476842624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828546.762509:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828546/real 1787828546] req@ffff9a78841da940 x1874671476842112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828546.762541:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476842112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828546.762559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828546.762566:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828546.762573:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476842112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828546.762640:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828546.762665:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828546.762677:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828546/real 1787828546] req@ffff9a78841d9680 x1874671476842240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828546.762688:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476842240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828546.762696:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828546.762699:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828546.762702:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828546.762708:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828546.762722:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828546.762730:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828546/real 1787828546] req@ffff9a78841d92c0 x1874671476842368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828546.762740:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476842368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828546.762746:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828546.762749:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828546.762751:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828546.762755:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828546.762768:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828546.762775:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828546/real 1787828546] req@ffff9a78841db840 x1874671476842496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828546.762784:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476842496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828546.762793:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828546.762795:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828546.762798:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828546.762802:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828546.762814:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828546.762821:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828546/real 1787828546] req@ffff9a78841d8b40 x1874671476842624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828546.762858:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476842624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828546.762868:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828546.762870:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828546.762873:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828547.786125:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828548.810140:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828549.834123:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828550.857992:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828551.881947:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828551.881964:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828551.881971:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828551.881986:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2919 00010000:00010000:0.0:1787828551.881987:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828551.882013:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828551.882024:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828551.882082:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476842752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828551.882112:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828551.882116:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828551.882118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828551.882122:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2919 00000100:00080000:2.0:1787828551.882126:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828551.882133:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828551.882157:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828551.882166:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476842752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828551.882173:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476842880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828551.882184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828551.882187:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828551.882189:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828551.882193:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2919 00000100:00080000:2.0:1787828551.882196:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828551.882199:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828551.882202:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828551.882215:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476843008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828551.882222:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828551.882225:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828551.882227:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828551.882230:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2919 00000100:00080000:2.0:1787828551.882245:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828551.882251:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828551.882269:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828551.882271:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828551.882295:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476843136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828551.882310:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828551.882315:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828551.882318:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828551.882336:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2919 00000100:00080000:2.0:1787828551.882343:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828551.882347:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828551/real 1787828551] req@ffff9a78841d9e00 x1874671476842752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828551.882354:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828551.882368:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828551.882378:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476842752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828551.882387:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476843264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828551.882412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828551.882420:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828551.882426:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476842752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828551.882474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828551.882496:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828551.882508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828551/real 1787828551] req@ffff9a78841db480 x1874671476842880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828551.882519:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476842880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828551.882533:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828551.882536:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828551.882539:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476842880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828551.882543:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828551.882557:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828551.882564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828551/real 1787828551] req@ffff9a78841d83c0 x1874671476843008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828551.882573:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476843008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828551.882581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828551.882583:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828551.882585:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828551.882589:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828551.882612:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828551.882620:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828551/real 1787828551] req@ffff9a78841d8000 x1874671476843136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828551.882629:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476843136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828551.882638:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828551.882640:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828551.882644:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828551.882648:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828551.882661:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828551.882668:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828551/real 1787828551] req@ffff9a78841da580 x1874671476843264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828551.882677:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476843264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828551.882686:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828551.882688:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828551.882690:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828552.906071:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828553.929954:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828554.954066:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828555.977918:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828557.002006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828557.002025:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828557.002040:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828557.002046:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828557.002060:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2924 00000100:00080000:0.0:1787828557.002075:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.002085:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828557.002130:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476843392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828557.002161:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828557.002165:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828557.002168:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828557.002172:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2924 00000100:00080000:0.0:1787828557.002178:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.002183:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828557.002194:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828557.002213:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476843392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828557.002215:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476843520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828557.002236:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828557.002240:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828557.002241:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828557.002246:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2924 00000100:00080000:0.0:1787828557.002249:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.002252:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828557.002258:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828557.002271:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476843648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828557.002281:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828557.002302:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828557.002304:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828557.002309:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2924 00000800:00000100:3.0:1787828557.002317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828557.002318:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.002329:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828557.002337:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828557.002353:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476843776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828557.002414:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828557.002418:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828557.002425:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828557.002425:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff9a7890a3da40 x1874671476843392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828557.002441:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2924 00000100:00080000:0.0:1787828557.002467:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828557.002477:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828557.002485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476843392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828557.002486:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828557.002512:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787828557.002520:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476843904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828557.002523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828557.002537:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476843392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828557.002619:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828557.002652:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828557.002669:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff9a7890a3de00 x1874671476843520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828557.002684:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476843520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828557.002695:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828557.002698:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828557.002702:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828557.002709:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828557.002727:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828557.002738:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff9a7890a3f480 x1874671476843648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828557.002751:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476843648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828557.002760:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828557.002763:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828557.002766:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828557.002773:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828557.002791:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828557.002801:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff9a7890a3c3c0 x1874671476843776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828557.002813:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476843776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828557.002825:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828557.002829:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828557.002863:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828557.002870:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828557.002891:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828557.002904:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828557/real 1787828557] req@ffff9a7890a3c000 x1874671476843904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828557.002917:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476843904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828557.002928:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828557.002949:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828557.002953:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476843904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828558.026032:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828559.050077:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828560.074018:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828561.097992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828562.121944:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828562.121952:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828562.121956:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828562.121965:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2929 00000100:00080000:0.0:1787828562.121973:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828562.121979:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828562.122008:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828562.122014:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476844032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828562.122035:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828562.122037:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828562.122039:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828562.122043:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2929 00000100:00080000:0.0:1787828562.122046:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828562.122049:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828562.122057:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828562.122067:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476844032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828562.122069:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476844160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828562.122090:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828562.122092:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828562.122094:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828562.122097:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2929 00000100:00080000:0.0:1787828562.122099:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828562.122102:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828562.122105:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828562.122118:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476844288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828562.122128:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828562.122133:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828562.122134:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828562.122140:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2929 00000800:00000100:3.0:1787828562.122143:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828562.122147:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828562.122154:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828562.122161:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828562.122173:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476844416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828562.122181:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828562.122185:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787828562.122186:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff9a7890a3fc00 x1874671476844032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828562.122200:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828562.122209:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2929 00000100:00080000:0.0:1787828562.122214:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828562.122217:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828562.122221:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476844032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828562.122222:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828562.122236:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476844544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828562.122236:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828562.122244:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828562.122252:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476844032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828562.122273:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828562.122288:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828562.122296:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff9a7890a3c000 x1874671476844160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828562.122304:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476844160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828562.122309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828562.122311:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828562.122313:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828562.122316:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828562.122342:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828562.122349:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff9a7890a3c3c0 x1874671476844288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828562.122368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476844288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828562.122373:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828562.122375:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828562.122377:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828562.122381:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828562.122392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828562.122398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff9a7890a3f480 x1874671476844416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828562.122405:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476844416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828562.122411:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828562.122412:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828562.122414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828562.122418:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828562.122436:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828562.122441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828562/real 1787828562] req@ffff9a7890a3de00 x1874671476844544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828562.122449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476844544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828562.122454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828562.122456:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828562.122458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828563.146125:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828564.170042:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828565.193992:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828566.218055:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828567.241929:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828567.241940:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828567.241945:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828567.241956:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2934 00000100:00080000:2.0:1787828567.241968:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828567.241976:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828567.241994:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828567.242015:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476844672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828567.242063:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828567.242068:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828567.242072:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828567.242078:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2934 00000100:00080000:2.0:1787828567.242085:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828567.242093:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828567.242107:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828567.242119:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476844672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828567.242131:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476844800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828567.242150:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828567.242156:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828567.242159:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828567.242166:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2934 00000100:00080000:2.0:1787828567.242197:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828567.242202:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828567.242206:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828567.242211:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828567.242228:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476844928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828567.242239:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828567.242243:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828567.242246:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828567.242250:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2934 00000100:00080000:2.0:1787828567.242255:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828567.242261:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff9a78841d8780 x1874671476844672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828567.242262:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828567.242272:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828567.242301:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476844672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828567.242305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476845056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828567.242319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828567.242326:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828567.242326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828567.242331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476844672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828567.242336:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828567.242340:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828567.242346:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2934 00000100:00080000:2.0:1787828567.242351:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828567.242355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844800:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787828567.242360:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828567.242368:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787828567.242381:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828567.242386:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476845184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828567.242395:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff9a78841d9a40 x1874671476844800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828567.242410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476844800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828567.242416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828567.242420:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828567.242422:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828567.242427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828567.242438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828567.242445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff9a78841da1c0 x1874671476844928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828567.242452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476844928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828567.242458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828567.242459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828567.242462:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476844928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828567.242466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828567.242476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828567.242481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff9a78841db0c0 x1874671476845056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828567.242489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476845056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828567.242495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828567.242497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828567.242499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828567.242517:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828567.242528:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828567.242535:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828567/real 1787828567] req@ffff9a78841dbc00 x1874671476845184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828567.242543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476845184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828567.242549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828567.242551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828567.242553:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828568.266044:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828569.289971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828570.314039:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828571.338033:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828572.362028:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828572.362043:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828572.362050:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828572.362065:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2939 00000100:00080000:3.0:1787828572.362081:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828572.362091:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828572.362197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476845312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828572.362268:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828572.362273:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828572.362276:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828572.362297:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2939 00000100:00080000:3.0:1787828572.362302:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828572.362307:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828572.362321:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828572.362338:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476845440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828572.362351:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828572.362355:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828572.362359:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828572.362365:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476845312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828572.362366:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2939 00000100:00080000:3.0:1787828572.362375:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828572.362380:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828572.362385:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828572.362398:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476845568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828572.362415:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828572.362419:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828572.362421:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828572.362425:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2939 00000100:00080000:3.0:1787828572.362453:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787828572.362468:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828572.362471:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828572.362483:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828572.362506:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476845696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828572.362521:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828572.362525:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828572.362527:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828572.362533:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2939 00000100:00080000:3.0:1787828572.362538:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828572.362544:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828572.362550:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828572.362564:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476845824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828572.362585:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9a78a6775680 x1874671476845312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828572.362618:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476845312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828572.362632:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828572.362638:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828572.362643:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476845312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828572.362682:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845440:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787828572.362690:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:0.0:1787828572.362731:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828572.362750:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9a78a6776d00 x1874671476845440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828572.362762:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476845440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828572.362780:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828572.362783:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828572.362787:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828572.362793:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828572.362811:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828572.362820:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9a78a67761c0 x1874671476845568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828572.362901:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476845568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828572.362911:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828572.362914:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828572.362917:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828572.362924:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828572.362948:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828572.362960:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9a78a6774780 x1874671476845696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828572.362970:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476845696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828572.362992:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828572.362995:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828572.362999:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828572.363004:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828572.363021:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828572.363030:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828572/real 1787828572] req@ffff9a78a67770c0 x1874671476845824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828572.363041:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476845824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828572.363052:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828572.363055:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828572.363058:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476845824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828573.386055:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828574.409973:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828575.434004:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828576.458032:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828577.481962:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828577.481978:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828577.481984:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828577.481998:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2944 00000100:00080000:3.0:1787828577.482013:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828577.482023:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828577.482075:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476845952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828577.482110:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828577.482114:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828577.482117:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828577.482121:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2944 00000100:00080000:3.0:1787828577.482124:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828577.482128:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828577.482141:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828577.482155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476846080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828577.482167:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476845952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828577.482169:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828577.482180:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828577.482183:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828577.482189:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2944 00000100:00080000:3.0:1787828577.482192:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828577.482195:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828577.482201:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828577.482231:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476846208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828577.482244:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828577.482248:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828577.482251:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828577.482266:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2944 00000100:00080000:3.0:1787828577.482288:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828577.482292:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828577.482298:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828577.482306:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828577.482336:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476846336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828577.482351:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828577.482355:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828577.482358:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828577.482362:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2944 00000100:00080000:3.0:1787828577.482367:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828577.482372:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828577.482379:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828577.482397:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476846464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828577.482410:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828577/real 1787828577] req@ffff9a78a6776940 x1874671476845952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828577.482467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476845952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828577.482485:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828577.482491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828577.482498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476845952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828577.482544:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828577.482609:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:1.0:1787828577.482617:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828577.482647:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828577/real 1787828577] req@ffff9a78a6776580 x1874671476846080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828577.482665:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476846080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828577.482674:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828577.482677:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828577.482681:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828577.482687:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828577.482710:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828577.482720:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828577/real 1787828577] req@ffff9a78a6774b40 x1874671476846208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828577.482731:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476846208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828577.482750:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828577.482754:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828577.482757:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828577.482763:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828577.482781:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828577.482791:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828577/real 1787828577] req@ffff9a78a67752c0 x1874671476846336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828577.482809:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476846336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828577.482819:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828577.482822:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828577.482826:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828577.482860:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828577.482879:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828577.482912:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828577/real 1787828577] req@ffff9a78a6775a40 x1874671476846464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828577.482927:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476846464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828577.482940:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828577.482944:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828577.482948:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828578.506051:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828579.530105:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828580.554032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828581.578056:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828582.602026:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828582.602038:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828582.602043:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828582.602048:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828582.602056:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2950 00000100:00080000:2.0:1787828582.602069:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828582.602077:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828582.602126:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476846592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828582.602164:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828582.602167:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828582.602169:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828582.602172:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2950 00000100:00080000:2.0:1787828582.602175:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828582.602179:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828582.602201:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828582.602213:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476846720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828582.602222:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828582.602226:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828582.602229:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828582.602236:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476846592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828582.602237:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2950 00000100:00080000:2.0:1787828582.602246:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828582.602255:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828582.602259:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828582.602269:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476846848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828582.602277:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828582.602280:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828582.602282:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828582.602286:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2950 00000100:00080000:2.0:1787828582.602307:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828582.602310:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828582.602314:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787828582.602322:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828582.602332:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476846976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828582.602345:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828582.602348:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828582.602350:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828582.602355:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2950 00000100:00080000:2.0:1787828582.602361:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828582.602367:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828582.602371:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828582.602381:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828582/real 1787828582] req@ffff9a78841dbc00 x1874671476846592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828582.602382:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476847104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828582.602448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476846592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828582.602462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828582.602466:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828582.602471:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476846592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828582.602494:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828582.602518:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828582.602534:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828582/real 1787828582] req@ffff9a78841db0c0 x1874671476846720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828582.602550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476846720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828582.602560:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828582.602564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828582.602568:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828582.602577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828582.602604:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828582.602620:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828582/real 1787828582] req@ffff9a78841da1c0 x1874671476846848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828582.602635:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476846848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828582.602646:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828582.602651:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828582.602656:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828582.602664:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828582.602690:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828582.602707:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828582/real 1787828582] req@ffff9a78841d9a40 x1874671476846976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828582.602722:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476846976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828582.602736:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828582.602740:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828582.602744:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476846976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828582.602752:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828582.602780:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828582.602796:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828582/real 1787828582] req@ffff9a78841d8780 x1874671476847104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828582.602812:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476847104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828582.602826:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828582.602872:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828582.602877:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828583.626041:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828584.650044:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828585.674100:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828586.698079:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828587.722052:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787828587.722064:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828587.722065:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828587.722076:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828587.722091:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2955 00000100:00080000:0.0:1787828587.722105:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828587.722114:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828587.722192:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476847232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828587.722233:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828587.722238:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828587.722241:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828587.722245:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2955 00000100:00080000:0.0:1787828587.722250:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828587.722254:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828587.722267:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828587.722285:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476847360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828587.722302:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828587.722303:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476847232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828587.722314:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828587.722320:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828587.722327:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2955 00000100:00080000:0.0:1787828587.722331:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828587.722335:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828587.722342:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828587.722359:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476847488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828587.722380:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828587.722385:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828587.722387:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828587.722391:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2955 00000100:00080000:0.0:1787828587.722396:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828587.722400:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828587.722406:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828587.722424:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476847616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828587.722436:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828587.722441:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828587.722444:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828587.722450:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2955 00000100:00080000:0.0:1787828587.722454:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828587.722460:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828587.722467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847360:192.168.203.140@tcp:38:kworker.0 00000008:00100000:0.0:1787828587.722470:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828587.722510:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476847744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828587.722535:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828587.722569:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828587.722604:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828588.745965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828589.770002:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828590.794117:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828590.794242:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787828590.794278:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828590.794291:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828590.794327:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828590.794334:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828590.794341:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828590.794348:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787828590.794368:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828587/real 1787828590] req@ffff9a7890a3c000 x1874671476847744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828590.794404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476847744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828590.794419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828590.794424:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828590.794430:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847744:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828590.794438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828587/real 1787828590] req@ffff9a7890a3de00 x1874671476847360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828590.794449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476847360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828590.794458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828590.794462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828590.794465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828590.794473:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828587/real 1787828590] req@ffff9a7890a3f480 x1874671476847488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828590.794483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476847488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828590.794490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828590.794495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828590.794498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828590.794504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828587/real 1787828590] req@ffff9a7890a3c3c0 x1874671476847616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828590.794517:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476847616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828590.794527:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828590.794531:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828590.794534:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476847616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828590.794541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828587/real 1787828590] req@ffff9a7890a3da40 x1874671476847232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828590.794552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476847232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828590.794560:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828590.794563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828590.794566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476847232:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787828591.818001:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828592.841979:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828592.841988:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828592.841990:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828592.841996:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828592.842007:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2960 00000100:00080000:2.0:1787828592.842019:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828592.842026:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828592.842065:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476847872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828592.842089:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828592.842094:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828592.842096:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828592.842104:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2960 00000100:00080000:2.0:1787828592.842120:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828592.842126:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828592.842138:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476847872:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828592.842139:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828592.842160:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476848000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828592.842173:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828592.842177:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828592.842180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828592.842185:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2960 00000100:00080000:2.0:1787828592.842189:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828592.842195:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828592.842201:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828592.842216:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476848128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828592.842227:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787828592.842229:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828592.842235:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828592.842238:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828592.842244:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2960 00000100:00080000:2.0:1787828592.842249:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828592.842255:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828592.842262:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828592.842283:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476848256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828592.842296:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828592.842300:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828592.842302:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828592.842310:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2960 00000100:00100000:3.0:1787828592.842311:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9a78841da580 x1874671476847872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828592.842315:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828592.842324:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828592.842331:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828592.842353:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476848384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828592.842354:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476847872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828592.842387:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828592.842394:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828592.842400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476847872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828592.842435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828592.842462:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828592.842476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9a78841d8000 x1874671476848000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828592.842488:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476848000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828592.842496:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828592.842500:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828592.842503:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828592.842509:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828592.842528:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828592.842540:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9a78841d83c0 x1874671476848128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828592.842551:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476848128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828592.842559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828592.842562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828592.842566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828592.842572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828592.842592:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828592.842605:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9a78841db480 x1874671476848256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828592.842617:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476848256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828592.842627:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828592.842630:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828592.842634:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828592.842640:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828592.842660:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828592.842672:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9a78841d9e00 x1874671476848384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828592.843148:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476848384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828592.843161:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828592.843165:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828592.843169:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828593.866004:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828594.889979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828595.913985:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828596.938055:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828597.961975:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828597.961987:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828597.961993:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828597.962006:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2965 00000100:00080000:2.0:1787828597.962021:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828597.962030:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828597.962043:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828597.962097:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476848512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828597.962149:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828597.962152:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828597.962154:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828597.962158:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2965 00000100:00080000:2.0:1787828597.962165:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828597.962169:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828597.962181:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828597.962193:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476848640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828597.962201:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476848512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828597.962204:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828597.962215:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828597.962218:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828597.962223:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2965 00000100:00080000:2.0:1787828597.962225:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828597.962227:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828597.962232:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828597.962245:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476848768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828597.962251:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828597.962255:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828597.962256:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828597.962260:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2965 00000100:00080000:2.0:1787828597.962274:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828597.962278:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828597.962283:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828597.962291:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828597.962300:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476848896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828597.962310:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828597.962313:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828597.962315:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828597.962319:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2965 00000100:00080000:2.0:1787828597.962322:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828597.962325:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828597.962330:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828597.962347:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476849024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828597.962352:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9a78841d8b40 x1874671476848512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828597.962403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476848512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828597.962415:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828597.962418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828597.962421:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476848512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828597.962442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828597.962458:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828597.962467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9a78841db840 x1874671476848640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828597.962475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476848640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828597.962481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828597.962483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828597.962485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828597.962489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828597.962499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828597.962505:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9a78841d92c0 x1874671476848768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828597.962520:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476848768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828597.962531:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828597.962533:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828597.962535:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828597.962540:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828597.962552:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828597.962559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9a78841d9680 x1874671476848896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828597.962566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476848896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828597.962573:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828597.962575:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828597.962576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476848896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828597.962580:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828597.962590:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828597.962596:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9a78841da940 x1874671476849024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828597.962603:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476849024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828597.962609:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828597.962611:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828597.962614:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828598.986006:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828600.009963:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828601.033989:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828602.058001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828603.081958:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828603.081974:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828603.081979:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828603.081993:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2970 00000100:00080000:2.0:1787828603.082009:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828603.082019:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828603.082022:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828603.082083:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476849152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828603.082117:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828603.082121:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828603.082123:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828603.082128:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2970 00000100:00080000:2.0:1787828603.082132:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828603.082138:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828603.082151:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828603.082164:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476849152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828603.082175:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476849280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828603.082199:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828603.082203:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828603.082205:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828603.082211:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2970 00000100:00080000:2.0:1787828603.082226:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828603.082232:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828603.082237:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828603.082247:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828603.082269:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476849408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828603.082282:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828603.082285:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff9a78841da940 x1874671476849152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828603.082290:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828603.082296:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828603.082311:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2970 00000100:00100000:3.0:1787828603.082317:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476849152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828603.082320:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828603.082335:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828603.082335:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828603.082343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:2.0:1787828603.082348:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828603.082349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476849152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828603.082375:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476849536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828603.082384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849280:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787828603.082408:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828603.082414:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828603.082420:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828603.082420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828603.082430:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2970 00000100:00100000:3.0:1787828603.082437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff9a78841d9680 x1874671476849280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828603.082439:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828603.082456:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828603.082457:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476849280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787828603.082474:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828603.082474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828603.082490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828603.082493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828603.082500:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828603.082507:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476849664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828603.082519:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828603.082535:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff9a78841d92c0 x1874671476849408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828603.082543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476849408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828603.082551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828603.082554:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828603.082556:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828603.082566:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828603.082577:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828603.082583:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff9a78841db840 x1874671476849536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828603.082591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476849536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828603.082597:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828603.082599:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828603.082601:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828603.082605:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828603.082614:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828603.082620:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828603/real 1787828603] req@ffff9a78841d8b40 x1874671476849664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828603.082626:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476849664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828603.082633:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828603.082635:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828603.082637:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828604.106009:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828605.130040:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828606.153994:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828607.177981:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828608.201961:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828608.201971:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828608.201976:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828608.201987:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2975 00000100:00080000:2.0:1787828608.202001:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828608.202021:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828608.202046:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828608.202071:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476849792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828608.202098:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828608.202101:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828608.202103:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828608.202106:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2975 00000100:00080000:2.0:1787828608.202109:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828608.202113:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828608.202123:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828608.202143:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476849920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828608.202149:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476849792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828608.202154:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828608.202167:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828608.202169:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828608.202173:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2975 00000100:00080000:2.0:1787828608.202175:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828608.202177:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828608.202181:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828608.202191:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476850048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828608.202198:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828608.202200:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828608.202202:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828608.202205:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2975 00000100:00080000:2.0:1787828608.202229:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828608.202232:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828608.202237:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828608.202241:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828608.202251:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476850176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828608.202260:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828608.202263:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828608.202265:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828608.202268:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2975 00000100:00080000:2.0:1787828608.202270:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828608.202272:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828608.202276:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828608.202283:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476850304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828608.202322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9a78841d9e00 x1874671476849792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828608.202342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476849792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828608.202357:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828608.202375:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828608.202379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476849792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828608.202400:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828608.202420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828608.202432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9a78841db480 x1874671476849920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828608.202445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476849920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828608.202454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828608.202457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828608.202461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476849920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828608.202469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828608.202489:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828608.202498:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9a78841d83c0 x1874671476850048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828608.202506:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476850048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828608.202512:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828608.202514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828608.202517:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828608.202521:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828608.202532:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828608.202538:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9a78841d8000 x1874671476850176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828608.202545:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476850176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828608.202552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828608.202554:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828608.202556:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828608.202560:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828608.202587:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828608.202594:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9a78841da580 x1874671476850304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828608.202602:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476850304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828608.202614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828608.202616:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828608.202618:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828609.226052:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828610.250036:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828611.273954:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828612.297984:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828613.321947:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828613.321956:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828613.321960:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828613.321979:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2980 00000100:00080000:0.0:1787828613.321992:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828613.322000:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828613.322034:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476850432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828613.322038:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828613.322060:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828613.322063:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828613.322065:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828613.322067:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2980 00000100:00080000:0.0:1787828613.322070:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828613.322076:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828613.322085:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828613.322094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476850560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828613.322104:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476850432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828613.322107:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828613.322122:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828613.322124:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828613.322128:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2980 00000100:00080000:0.0:1787828613.322130:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828613.322132:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828613.322136:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828613.322151:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476850688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828613.322157:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828613.322159:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828613.322160:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828613.322163:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2980 00000100:00080000:0.0:1787828613.322165:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828613.322167:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828613.322170:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828613.322176:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476850816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828613.322181:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828613.322184:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828613.322185:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828613.322188:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2980 00000100:00080000:0.0:1787828613.322198:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828613.322200:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828613.322203:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787828613.322212:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828613.322213:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476850944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828613.322286:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9a7890a3fc00 x1874671476850432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.322306:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476850432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.322316:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.322320:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828613.322323:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476850432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828613.322349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828613.322366:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828613.322375:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9a7890a3da40 x1874671476850560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.322383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476850560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.322394:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.322396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828613.322399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828613.322403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828613.322413:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828613.322419:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9a7890a3c3c0 x1874671476850688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.322426:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476850688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.322432:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.322434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828613.322436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828613.322439:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828613.322450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828613.322456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9a7890a3f480 x1874671476850816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.322464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476850816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.322470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.322472:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828613.322475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828613.322478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828613.322487:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828613.322493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9a7890a3de00 x1874671476850944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.322499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476850944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.322506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.322508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828613.322510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476850944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828614.346068:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828615.369979:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828616.393964:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828617.417959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828618.441955:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787828618.441960:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828618.441968:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828618.441973:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.441983:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2985 00000100:00080000:1.0:1787828618.441995:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.442002:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828618.442037:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476851072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828618.442065:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828618.442069:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828618.442070:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.442074:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2985 00000100:00080000:1.0:1787828618.442078:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.442082:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828618.442092:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828618.442109:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476851200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828618.442109:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476851072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828618.442125:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828618.442129:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828618.442131:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.442136:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2985 00000100:00080000:1.0:1787828618.442140:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.442144:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828618.442150:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828618.442163:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476851328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828618.442169:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828618.442181:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828618.442194:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828618.442197:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.442203:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2985 00000100:00080000:1.0:1787828618.442207:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.442212:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828618.442219:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828618.442220:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9a7890b1da40 x1874671476851072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828618.442238:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476851456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828618.442240:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476851072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.442251:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.442261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787828618.442263:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828618.442269:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476851072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828618.442270:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828618.442275:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828618.442283:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2985 00000100:00080000:1.0:1787828618.442287:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828618.442292:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828618.442297:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828618.442300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787828618.442314:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476851584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828618.442326:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828618.442341:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9a7890b1e940 x1874671476851200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828618.442352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476851200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.442359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.442361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828618.442363:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828618.442368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828618.442385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828618.442396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9a7890b1de00 x1874671476851328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828618.442410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476851328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.442419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.442423:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828618.442426:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828618.442432:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828618.442448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828618.442456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9a7890b1d680 x1874671476851456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828618.442464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476851456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.442471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.442473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828618.442475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828618.442486:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828618.442497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828618.442503:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9a7890b1e580 x1874671476851584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828618.442510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476851584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.442517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.442518:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828618.442520:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828619.465950:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828620.489978:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828621.513952:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828622.537952:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828623.561960:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828623.561960:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828623.561972:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828623.561978:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828623.561991:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2991 00000100:00080000:0.0:1787828623.562005:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828623.562013:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828623.562063:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476851712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828623.562092:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828623.562097:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828623.562099:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828623.562104:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2991 00000100:00080000:0.0:1787828623.562109:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828623.562114:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828623.562125:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828623.562128:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476851712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828623.562141:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476851840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828623.562151:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828623.562154:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828623.562155:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828623.562158:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2991 00000100:00080000:0.0:1787828623.562161:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828623.562163:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828623.562167:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828623.562177:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476851968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828623.562184:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828623.562187:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828623.562188:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828623.562190:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2991 00000100:00080000:0.0:1787828623.562200:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828623.562203:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828623.562206:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828623.562210:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828623.562216:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476852096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828623.562223:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828623.562226:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828623.562227:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828623.562231:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2991 00000100:00080000:0.0:1787828623.562237:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828623.562239:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828623.562242:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828623.562251:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476852224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828623.562272:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9a7890a3c000 x1874671476851712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.562293:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476851712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.562303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.562306:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828623.562309:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476851712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828623.562333:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828623.562353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828623.562366:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9a7890a3de00 x1874671476851840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.562380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476851840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.562390:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.562393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828623.562397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828623.562403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828623.562426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828623.562435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9a7890a3f480 x1874671476851968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.562443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476851968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.562449:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.562451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828623.562453:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476851968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828623.562456:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828623.562467:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828623.562474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9a7890a3c3c0 x1874671476852096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.562481:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476852096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.562488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.562490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828623.562492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828623.562495:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828623.562505:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828623.562510:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9a7890a3da40 x1874671476852224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.562518:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476852224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.562524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.562526:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828623.562527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828624.586064:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828625.610037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828626.634041:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828627.658075:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828628.681963:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828628.681974:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828628.681979:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828628.681990:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 2996 00000100:00080000:1.0:1787828628.682002:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828628.682010:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828628.682011:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828628.682056:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476852352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828628.682080:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828628.682083:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828628.682085:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828628.682088:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2996 00000100:00080000:1.0:1787828628.682091:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828628.682094:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828628.682104:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828628.682116:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476852480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828628.682116:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476852352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828628.682131:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828628.682133:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828628.682134:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828628.682137:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2996 00000100:00080000:1.0:1787828628.682141:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828628.682144:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828628.682147:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828628.682156:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476852608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828628.682163:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828628.682166:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828628.682167:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828628.682170:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2996 00000100:00080000:1.0:1787828628.682185:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828628.682188:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828628.682191:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828628.682196:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828628.682201:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476852736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828628.682208:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828628.682211:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828628.682213:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828628.682216:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 2996 00000100:00080000:1.0:1787828628.682223:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828628.682225:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828628.682228:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828628.682236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476852864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828628.682253:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9a7890b1fc00 x1874671476852352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.682271:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476852352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.682280:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.682284:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828628.682288:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476852352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828628.682315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828628.682339:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828628.682354:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9a7890b1c000 x1874671476852480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.682367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476852480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.682375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.682379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828628.682382:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828628.682389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828628.682411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828628.682422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9a7890b1f0c0 x1874671476852608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.682430:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476852608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.682436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.682439:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828628.682441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828628.682446:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828628.682459:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828628.682465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9a7890b1f480 x1874671476852736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.682472:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476852736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.682479:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.682481:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828628.682483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828628.682486:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828628.682495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828628.682501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9a7890b1f840 x1874671476852864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.682509:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476852864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.682515:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.682517:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828628.682518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476852864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828629.706001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828630.729946:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828631.754009:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828632.777989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828633.801925:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.801934:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828633.801939:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828633.801951:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3001 00000100:00080000:2.0:1787828633.801963:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828633.801971:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828633.801981:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828633.802012:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476852992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828633.802035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.802038:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828633.802040:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828633.802043:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3001 00000100:00080000:2.0:1787828633.802047:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828633.802051:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828633.802060:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828633.802070:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476852992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828633.802074:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476853120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828633.802085:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.802087:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828633.802088:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828633.802092:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3001 00000100:00080000:2.0:1787828633.802095:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828633.802098:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828633.802102:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828633.802111:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476853248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828633.802126:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.802129:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828633.802141:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828633.802143:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828633.802148:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3001 00000100:00080000:2.0:1787828633.802152:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828633.802155:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828633.802160:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828633.802170:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476853376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828633.802176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.802178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828633.802180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828633.802182:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3001 00000100:00080000:2.0:1787828633.802186:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828633.802188:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828633.802192:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828633.802202:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9a78841d8780 x1874671476852992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828633.802203:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476853504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828633.802250:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476852992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.802260:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.802268:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828633.802272:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476852992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828633.802292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828633.802320:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828633.802330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9a78841d9a40 x1874671476853120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828633.802345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476853120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.802352:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.802354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828633.802356:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828633.802361:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828633.802373:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828633.802380:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9a78841da1c0 x1874671476853248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828633.802388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476853248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.802399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.802407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828633.802409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828633.802414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828633.802425:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828633.802432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9a78841db0c0 x1874671476853376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828633.802440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476853376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.802446:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.802448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828633.802450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828633.802456:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828633.802476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828633.802487:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9a78841dbc00 x1874671476853504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828633.802502:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476853504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.802513:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.802516:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828633.802519:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828634.825940:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828635.849989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828636.874133:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828637.897985:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828638.921987:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828638.921999:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828638.922004:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828638.922016:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3006 00000100:00080000:2.0:1787828638.922028:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828638.922036:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828638.922081:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476853632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828638.922096:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828638.922107:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828638.922112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828638.922114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828638.922118:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3006 00000100:00080000:2.0:1787828638.922121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828638.922125:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828638.922136:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828638.922150:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476853760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828638.922150:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476853632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828638.922170:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828638.922172:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828638.922173:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828638.922176:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3006 00000100:00080000:2.0:1787828638.922179:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828638.922182:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828638.922186:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828638.922209:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476853888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828638.922242:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828638.922245:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828638.922246:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828638.922249:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3006 00000100:00080000:2.0:1787828638.922273:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828638.922276:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828638.922280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828638.922289:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828638.922293:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476854016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828638.922304:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828638.922309:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828638.922311:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828638.922316:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3006 00000100:00080000:2.0:1787828638.922321:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828638.922325:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828638.922331:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828638.922343:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476854144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828638.922364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9a78841dbc00 x1874671476853632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.922383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476853632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.922392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.922395:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828638.922399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476853632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828638.922434:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828638.922452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828638.922462:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9a78841db0c0 x1874671476853760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.922470:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476853760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.922476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.922478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828638.922480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828638.922484:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828638.922495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828638.922502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9a78841da1c0 x1874671476853888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.922510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476853888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.922515:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.922517:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828638.922519:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476853888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828638.922523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828638.922534:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828638.922540:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9a78841d9a40 x1874671476854016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.922559:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476854016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.922566:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.922567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828638.922569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828638.922573:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828638.922587:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828638.922598:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9a78841d8780 x1874671476854144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.922611:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476854144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.922633:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.922636:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828638.922640:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828639.946003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828640.970043:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828641.994029:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828643.018035:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828644.041924:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828644.041932:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828644.041936:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828644.041947:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3011 00000100:00080000:0.0:1787828644.041958:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828644.041966:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828644.041970:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828644.042004:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476854272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828644.042026:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828644.042029:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828644.042031:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828644.042034:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3011 00000100:00080000:0.0:1787828644.042036:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828644.042039:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828644.042052:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828644.042062:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476854400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828644.042070:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828644.042072:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828644.042074:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828644.042080:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3011 00000100:00080000:0.0:1787828644.042083:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828644.042083:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476854272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828644.042088:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828644.042096:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828644.042103:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476854528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828644.042110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828644.042113:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828644.042114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828644.042116:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3011 00000100:00080000:0.0:1787828644.042118:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828644.042120:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828644.042124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828644.042132:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476854656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828644.042138:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828644.042141:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828644.042142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828644.042144:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3011 00000100:00080000:0.0:1787828644.042147:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828644.042149:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828644.042152:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828644.042161:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476854784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828644.042166:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828644.042229:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9a7890a3fc00 x1874671476854272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.042248:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476854272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.042259:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.042263:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828644.042267:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476854272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828644.042275:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828644.042298:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828644.042310:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9a7890a3da40 x1874671476854400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.042322:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476854400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.042330:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.042334:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828644.042338:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828644.042378:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828644.042401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828644.042414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9a7890a3c3c0 x1874671476854528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.042427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476854528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.042436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.042440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828644.042443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828644.042450:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828644.042472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828644.042485:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9a7890a3f480 x1874671476854656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.042498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476854656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.042509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.042513:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828644.042516:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828644.042523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828644.042544:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828644.042556:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828644/real 1787828644] req@ffff9a7890a3de00 x1874671476854784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828644.042570:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476854784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828644.042581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828644.042585:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828644.042589:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476854784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828645.066023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828646.090037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828647.114008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828648.138061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828649.161952:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828649.161964:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828649.161969:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828649.161985:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3016 00000100:00080000:0.0:1787828649.161998:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828649.162007:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828649.162011:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828649.162053:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476854912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828649.162080:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828649.162083:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828649.162084:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828649.162087:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3016 00000100:00080000:0.0:1787828649.162090:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828649.162094:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828649.162103:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828649.162120:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476855040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828649.162128:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476854912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828649.162133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828649.162139:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828649.162140:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828649.162145:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3016 00000100:00080000:0.0:1787828649.162147:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828649.162152:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828649.162156:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828649.162165:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476855168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828649.162172:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828649.162175:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828649.162177:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828649.162180:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3016 00000100:00080000:0.0:1787828649.162191:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828649.162197:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828649.162204:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787828649.162208:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828649.162228:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476855296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828649.162237:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828649.162240:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828649.162242:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828649.162246:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3016 00000100:00080000:0.0:1787828649.162250:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828649.162253:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828649.162259:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9a7890a3c000 x1874671476854912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828649.162263:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828649.162286:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476854912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828649.162292:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476855424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828649.162310:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.162316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828649.162321:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476854912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828649.162346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828649.162365:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828649.162375:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9a7890a3de00 x1874671476855040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828649.162383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476855040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828649.162393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.162395:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828649.162397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828649.162401:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828649.162411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828649.162417:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9a7890a3f480 x1874671476855168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828649.162423:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476855168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828649.162428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.162430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828649.162432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828649.162436:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828649.162446:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828649.162451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9a7890a3c3c0 x1874671476855296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828649.162458:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476855296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828649.162464:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.162466:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828649.162468:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828649.162471:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828649.162480:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828649.162486:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828649/real 1787828649] req@ffff9a7890a3da40 x1874671476855424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828649.162492:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476855424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828649.162498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828649.162500:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828649.162502:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828650.186041:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828651.210070:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828652.233997:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828653.257965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828654.281909:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.281918:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828654.281921:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.281930:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3021 00000100:00080000:0.0:1787828654.281939:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.281945:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828654.281970:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828654.281979:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476855552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828654.282003:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.282006:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828654.282008:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.282011:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3021 00000100:00080000:0.0:1787828654.282013:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.282016:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828654.282024:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828654.282037:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476855680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828654.282038:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476855552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828654.282052:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.282055:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828654.282056:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.282059:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3021 00000100:00080000:0.0:1787828654.282061:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.282063:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828654.282066:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828654.282075:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476855808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828654.282090:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.282093:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828654.282095:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.282098:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3021 00000100:00080000:0.0:1787828654.282101:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.282103:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828654.282107:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828654.282116:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476855936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828654.282122:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.282124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828654.282126:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.282129:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3021 00000100:00080000:0.0:1787828654.282131:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.282135:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828654.282138:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828654.282146:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476856064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828654.282152:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828654.282188:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828654.282213:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828654.282240:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828655.306017:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828656.329989:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828657.354061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787828657.354124:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787828657.354161:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787828657.354174:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828657.354213:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828657.354220:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828657.354225:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787828657.354231:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787828657.354275:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828657] req@ffff9a7890a3de00 x1874671476856064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828657.354312:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476856064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828657.354336:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828657.354342:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828657.354347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856064:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828657.354357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828657] req@ffff9a7890a3da40 x1874671476855680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828657.354371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476855680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828657.354380:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828657.354384:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828657.354387:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828657.354394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828657] req@ffff9a7890a3c3c0 x1874671476855808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828657.354406:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476855808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828657.354414:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828657.354418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828657.354421:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828657.354429:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828657] req@ffff9a7890a3f480 x1874671476855936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828657.354441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476855936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828657.354459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828657.354463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828657.354466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476855936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828657.354474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828657] req@ffff9a7890a3fc00 x1874671476855552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828657.354495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476855552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828657.354503:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828657.354507:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828657.354511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476855552:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787828658.378202:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828659.401928:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828659.401936:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828659.401940:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828659.401948:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3026 00000100:00080000:1.0:1787828659.401965:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828659.401971:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828659.402001:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476856192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828659.402021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828659.402022:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828659.402027:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828659.402029:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828659.402032:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3026 00000100:00080000:1.0:1787828659.402037:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828659.402040:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828659.402049:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828659.402060:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476856320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828659.402060:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476856192:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828659.402094:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828659.402098:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828659.402100:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828659.402105:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3026 00000100:00080000:1.0:1787828659.402108:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828659.402112:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828659.402117:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828659.402131:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476856448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828659.402141:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787828659.402147:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828659.402148:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828659.402152:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828659.402164:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3026 00000100:00080000:1.0:1787828659.402169:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828659.402174:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828659.402178:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828659.402189:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476856576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828659.402201:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828659.402204:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828659.402207:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828659.402213:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3026 00000100:00100000:3.0:1787828659.402214:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9a7890b1f840 x1874671476856192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828659.402218:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828659.402226:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828659.402236:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828659.402247:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476856192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828659.402253:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476856704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828659.402273:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828659.402280:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828659.402286:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476856192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828659.402315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828659.402347:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828659.402362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9a7890b1f480 x1874671476856320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828659.402376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476856320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828659.402385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828659.402388:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828659.402392:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828659.402398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828659.402420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828659.402432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9a7890b1f0c0 x1874671476856448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828659.402444:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476856448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828659.402451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828659.402454:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828659.402458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828659.402463:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828659.402483:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828659.402496:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9a7890b1c000 x1874671476856576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828659.402514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476856576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828659.402525:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828659.402528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828659.402532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828659.402538:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828659.402559:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828659.402571:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9a7890b1fc00 x1874671476856704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828659.402583:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476856704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828659.402594:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828659.402598:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828659.402601:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828660.426220:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828661.450018:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828662.473947:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828663.498033:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828664.521924:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828664.521931:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828664.521935:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828664.521943:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3032 00000100:00080000:1.0:1787828664.521952:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828664.521958:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828664.521980:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828664.521995:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476856832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828664.522021:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828664.522025:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828664.522027:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828664.522031:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3032 00000100:00080000:1.0:1787828664.522034:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828664.522039:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828664.522047:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828664.522055:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476856832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828664.522060:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476856960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828664.522070:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828664.522075:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828664.522077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828664.522081:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3032 00000100:00080000:1.0:1787828664.522085:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828664.522098:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828664.522108:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828664.522117:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828664.522124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476857088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828664.522136:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828664.522141:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828664.522143:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828664.522148:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3032 00000100:00080000:1.0:1787828664.522152:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828664.522156:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828664.522162:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828664.522168:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9a7890b1e580 x1874671476856832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828664.522177:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476857216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828664.522191:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476856832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828664.522192:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828664.522201:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828664.522204:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828664.522213:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3032 00000100:00080000:1.0:1787828664.522218:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828664.522223:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828664.522225:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787828664.522228:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828664.522231:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828664.522236:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476856832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828664.522240:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476857344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828664.522261:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828664.522279:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828664.522288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9a7890b1d680 x1874671476856960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828664.522296:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476856960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828664.522301:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828664.522303:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828664.522306:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476856960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828664.522309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828664.522318:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828664.522324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9a7890b1de00 x1874671476857088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828664.522331:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476857088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828664.522336:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828664.522338:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828664.522340:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828664.522344:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828664.522354:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828664.522364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9a7890b1e940 x1874671476857216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828664.522371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476857216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828664.522377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828664.522379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828664.522381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828664.522384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828664.522393:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828664.522398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9a7890b1da40 x1874671476857344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828664.522405:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476857344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828664.522411:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828664.522413:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828664.522414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828665.545982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828666.570049:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828667.593967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828668.618018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828669.641938:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828669.641949:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828669.641954:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828669.641962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828669.641965:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3037 00000100:00080000:0.0:1787828669.641978:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828669.641985:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828669.642032:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476857472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828669.642059:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828669.642063:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828669.642065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828669.642069:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3037 00000100:00080000:0.0:1787828669.642073:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828669.642078:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828669.642088:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828669.642090:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476857472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828669.642106:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476857600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828669.642119:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828669.642122:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828669.642125:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828669.642129:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3037 00000100:00080000:0.0:1787828669.642142:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828669.642147:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828669.642150:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787828669.642161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828669.642178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476857728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828669.642189:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828669.642194:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828669] req@ffff9a7890a3c000 x1874671476857472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828669.642196:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828669.642201:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828669.642211:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3037 00000100:00100000:3.0:1787828669.642216:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476857472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828669.642217:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828669.642227:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828669.642230:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787828669.642234:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787828669.642235:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828669.642241:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476857472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828669.642253:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476857856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828669.642263:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857600:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787828669.642270:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828669.642275:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828669.642277:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828669.642283:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828669.642285:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3037 00000100:00080000:0.0:1787828669.642292:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828669.642296:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828669] req@ffff9a7890a3fc00 x1874671476857600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828669.642301:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828669.642312:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828669.642326:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476857984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828669.642330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476857600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828669.642355:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828669.642359:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828669.642363:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828669.642372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828669.642392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828669.642404:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828669] req@ffff9a7890a3f480 x1874671476857728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828669.642415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476857728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828669.642429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828669.642433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828669.642436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828669.642450:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828669.642471:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828669.642481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828669] req@ffff9a7890a3c3c0 x1874671476857856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828669.642488:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476857856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828669.642495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828669.642497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828669.642499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828669.642503:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828669.642514:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828669.642520:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828669] req@ffff9a7890a3da40 x1874671476857984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828669.642527:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476857984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828669.642533:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828669.642534:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828669.642537:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476857984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828670.665994:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828671.689960:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828672.713968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828673.737972:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828674.761991:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828674.762007:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828674.762033:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828674.762047:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3042 00000100:00080000:0.0:1787828674.762061:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828674.762069:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828674.762127:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828674.762129:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476858112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828674.762164:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828674.762167:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828674.762169:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828674.762172:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3042 00000100:00080000:0.0:1787828674.762175:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828674.762179:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828674.762194:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828674.762213:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476858112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828674.762220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476858240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828674.762234:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828674.762238:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828674.762240:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828674.762246:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3042 00000100:00080000:0.0:1787828674.762251:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828674.762255:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828674.762262:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828674.762277:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476858368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828674.762285:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787828674.762290:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828674.762292:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828674.762295:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828674.762299:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3042 00000100:00080000:0.0:1787828674.762303:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828674.762307:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828674.762312:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828674.762323:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476858496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828674.762342:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828674.762346:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828674.762348:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828674.762354:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3042 00000100:00100000:3.0:1787828674.762358:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9a7890a3de00 x1874671476858112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828674.762360:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828674.762371:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828674.762378:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828674.762380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476858112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828674.762392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787828674.762396:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476858624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828674.762399:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828674.762407:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476858112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828674.762434:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828674.762450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828674.762459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9a7890a3da40 x1874671476858240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828674.762468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476858240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828674.762473:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828674.762475:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828674.762477:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828674.762481:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828674.762491:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828674.762497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9a7890a3c3c0 x1874671476858368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828674.762504:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476858368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828674.762510:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828674.762511:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828674.762513:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828674.762517:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828674.762526:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828674.762532:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9a7890a3f480 x1874671476858496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828674.762553:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476858496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828674.762561:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828674.762563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828674.762566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828674.762570:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828674.762582:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828674.762598:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9a7890a3fc00 x1874671476858624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828674.762605:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476858624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828674.762613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828674.762615:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828674.762617:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828675.786212:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828676.810030:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828677.834053:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828678.858054:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828679.881931:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828679.881941:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828679.881946:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828679.881967:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3047 00000100:00080000:0.0:1787828679.881978:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787828679.881982:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828679.881987:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828679.882026:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476858752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828679.882054:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828679.882059:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828679.882061:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828679.882065:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3047 00000100:00080000:0.0:1787828679.882069:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828679.882074:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828679.882092:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828679.882109:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476858880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828679.882116:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476858752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828679.882124:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828679.882129:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828679.882131:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828679.882136:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3047 00000100:00080000:0.0:1787828679.882141:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828679.882145:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828679.882151:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828679.882165:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476859008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828679.882176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828679.882180:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828679.882184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828679.882185:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828679.882193:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3047 00000100:00080000:0.0:1787828679.882199:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828679.882203:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828679.882209:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828679.882223:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476859136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828679.882234:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828679.882238:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828679.882249:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828679.882254:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828679/real 1787828679] req@ffff9a7890a3c000 x1874671476858752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828679.882258:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3047 00000100:00080000:0.0:1787828679.882265:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828679.882273:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828679.882280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828679.882286:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476858752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828679.882300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787828679.882305:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476859264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828679.882308:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828679.882317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476858752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828679.882346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828679.882370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828679.882385:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828679/real 1787828679] req@ffff9a7890a3fc00 x1874671476858880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828679.882397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476858880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828679.882406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828679.882410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828679.882414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476858880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828679.882420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828679.882437:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828679.882448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828679/real 1787828679] req@ffff9a7890a3f480 x1874671476859008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828679.882459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476859008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828679.882467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828679.882470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828679.882473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828679.882478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828679.882495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828679.882519:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828679/real 1787828679] req@ffff9a7890a3c3c0 x1874671476859136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828679.882543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476859136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828679.882552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828679.882555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828679.882558:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828679.882564:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828679.882582:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828679.882602:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828679/real 1787828679] req@ffff9a7890a3da40 x1874671476859264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828679.882613:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476859264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828679.882623:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828679.882626:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828679.882629:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828680.906013:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828681.930032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828682.953952:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828683.977924:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828685.001912:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787828685.001924:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828685.001924:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828685.001932:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828685.001960:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3052 00000100:00080000:1.0:1787828685.001971:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828685.001978:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828685.002013:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476859392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828685.002035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828685.002038:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828685.002039:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828685.002042:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3052 00000100:00080000:1.0:1787828685.002051:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828685.002054:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828685.002063:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828685.002071:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476859392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828685.002075:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476859520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828685.002086:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828685.002089:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828685.002090:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828685.002093:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3052 00000100:00080000:1.0:1787828685.002096:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828685.002098:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828685.002101:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828685.002111:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476859648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828685.002118:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828685.002121:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828685.002122:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828685.002125:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3052 00000100:00080000:1.0:1787828685.002129:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828685.002132:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828685.002135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828685.002139:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828685.002145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476859776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828685.002153:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828685.002156:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828685.002157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828685.002161:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3052 00000100:00080000:1.0:1787828685.002165:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828685.002167:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828685.002170:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828685.002178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476859904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828685.002197:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff9a7890b1da40 x1874671476859392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828685.002229:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476859392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828685.002238:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828685.002241:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828685.002244:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476859392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828685.002264:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828685.002283:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828685.002293:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff9a7890b1e940 x1874671476859520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828685.002302:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476859520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828685.002312:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828685.002315:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828685.002317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828685.002321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828685.002333:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828685.002339:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff9a7890b1de00 x1874671476859648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828685.002347:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476859648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828685.002353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828685.002355:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828685.002357:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828685.002360:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828685.002371:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828685.002377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff9a7890b1d680 x1874671476859776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828685.002385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476859776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828685.002391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828685.002393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828685.002396:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828685.002400:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828685.002409:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828685.002415:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828685/real 1787828685] req@ffff9a7890b1e580 x1874671476859904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828685.002423:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476859904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828685.002429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828685.002431:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828685.002433:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476859904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828686.025926:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828687.049951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828688.073925:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828689.097959:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828690.121936:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828690.121956:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828690.121960:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828690.121969:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3057 00000100:00080000:0.0:1787828690.121977:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828690.121984:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828690.121991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828690.122017:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476860032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828690.122045:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828690.122048:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828690.122049:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828690.122052:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3057 00000100:00080000:0.0:1787828690.122054:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828690.122057:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828690.122064:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828690.122074:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476860160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828690.122081:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828690.122086:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828690.122088:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828690.122088:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476860032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828690.122097:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3057 00000100:00080000:0.0:1787828690.122100:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828690.122102:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828690.122106:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828690.122114:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476860288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828690.122121:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828690.122124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828690.122125:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828690.122128:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3057 00000100:00080000:0.0:1787828690.122130:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828690.122133:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828690.122137:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828690.122144:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476860416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828690.122150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828690.122152:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828690.122153:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828690.122158:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3057 00000800:00000100:3.0:1787828690.122159:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828690.122162:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828690.122166:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828690.122170:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828690.122178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476860544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828690.122206:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff9a7890a3de00 x1874671476860032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828690.122220:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476860032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828690.122239:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828690.122242:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828690.122245:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476860032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828690.122262:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828690.122276:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828690.122284:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff9a7890a3da40 x1874671476860160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828690.122296:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476860160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828690.122304:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828690.122318:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828690.122322:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828690.122328:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828690.122349:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828690.122360:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff9a7890a3c3c0 x1874671476860288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828690.122368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476860288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828690.122374:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828690.122376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828690.122378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828690.122382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828690.122395:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828690.122401:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff9a7890a3f480 x1874671476860416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828690.122408:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476860416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828690.122414:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828690.122417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828690.122421:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828690.122427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828690.122460:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828690.122473:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828690/real 1787828690] req@ffff9a7890a3fc00 x1874671476860544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828690.122486:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476860544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828690.122493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828690.122496:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828690.122498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828691.145953:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828692.169960:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828693.193966:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828694.218006:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828695.242015:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828695.242025:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:2.0:1787828695.242027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828695.242030:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828695.242061:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3062 00000100:00080000:1.0:1787828695.242071:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828695.242078:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828695.242113:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476860672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828695.242159:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828695.242163:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828695.242165:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828695.242169:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3062 00000100:00080000:1.0:1787828695.242172:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828695.242177:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828695.242187:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828695.242216:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476860672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787828695.242220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476860800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828695.242241:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828695.242244:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828695.242245:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828695.242251:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3062 00000100:00080000:1.0:1787828695.242256:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828695.242259:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828695.242280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828695.242294:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476860928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828695.242303:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828695.242306:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828695.242307:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828695.242313:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3062 00000100:00080000:1.0:1787828695.242316:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828695.242320:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828695.242324:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828695.242326:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828695.242352:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476861056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828695.242361:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828695.242363:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828695.242365:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828695.242368:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3062 00000100:00080000:1.0:1787828695.242370:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828695.242373:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828695.242379:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828695.242423:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476861184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828695.242429:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff9a7890b1fc00 x1874671476860672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828695.242475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476860672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828695.242487:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828695.242493:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828695.242499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476860672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828695.242536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828695.242564:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828695.242576:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff9a7890b1c000 x1874671476860800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828695.242586:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476860800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828695.242593:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828695.242596:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828695.242598:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828695.242604:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828695.242617:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828695.242624:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff9a7890b1f0c0 x1874671476860928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828695.242632:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476860928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828695.242638:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828695.242640:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828695.242642:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476860928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828695.242647:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828695.242659:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828695.242666:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff9a7890b1f480 x1874671476861056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828695.242674:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476861056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828695.242682:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828695.242684:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828695.242686:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828695.242699:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828695.242712:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828695.242719:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828695/real 1787828695] req@ffff9a7890b1f840 x1874671476861184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828695.242727:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476861184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828695.242735:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828695.242738:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828695.242740:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828696.266118:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828697.290020:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828698.313997:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828699.338013:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828700.361948:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.361956:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828700.361960:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.361970:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3067 00000100:00080000:0.0:1787828700.361980:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828700.361986:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828700.362010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828700.362026:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476861312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828700.362056:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.362059:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828700.362060:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.362063:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3067 00000100:00080000:0.0:1787828700.362066:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828700.362070:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828700.362079:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828700.362093:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476861440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828700.362093:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476861312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828700.362108:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.362111:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828700.362113:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.362116:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3067 00000100:00080000:0.0:1787828700.362118:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828700.362121:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828700.362124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828700.362134:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476861568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828700.362140:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.362143:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828700.362144:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.362147:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3067 00000100:00080000:0.0:1787828700.362150:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828700.362152:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828700.362156:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828700.362165:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476861696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828700.362167:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828700.362175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828700.362178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828700.362180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828700.362183:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3067 00000100:00080000:0.0:1787828700.362193:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828700.362195:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828700.362198:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828700.362207:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476861824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828700.362215:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff9a7890a3c000 x1874671476861312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828700.362231:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476861312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828700.362240:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828700.362243:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828700.362247:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476861312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828700.362264:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828700.362279:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828700.362288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff9a7890a3fc00 x1874671476861440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828700.362295:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476861440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828700.362300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828700.362303:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828700.362305:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828700.362309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828700.362318:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828700.362324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff9a7890a3f480 x1874671476861568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828700.362330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476861568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828700.362336:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828700.362337:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828700.362340:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828700.362344:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828700.362354:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828700.362359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff9a7890a3c3c0 x1874671476861696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828700.362383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476861696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828700.362390:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828700.362392:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828700.362395:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828700.362399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828700.362412:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828700.362420:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828700/real 1787828700] req@ffff9a7890a3da40 x1874671476861824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828700.362427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476861824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828700.362433:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828700.362434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828700.362436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476861824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828701.386012:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828702.410026:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828703.433973:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828704.458009:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828705.481964:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828705.481975:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:1.0:1787828705.481980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828705.481981:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828705.481992:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3072 00000100:00080000:2.0:1787828705.482003:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828705.482010:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828705.482047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476861952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828705.482074:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828705.482078:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828705.482080:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828705.482085:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3072 00000100:00080000:2.0:1787828705.482089:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828705.482093:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828705.482101:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476861952:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828705.482103:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828705.482121:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476862080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828705.482133:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828705.482136:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828705.482138:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828705.482142:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3072 00000100:00080000:2.0:1787828705.482155:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828705.482160:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828705.482164:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828705.482171:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828705.482185:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476862208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828705.482201:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828705.482204:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828705.482206:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828705.482216:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3072 00000100:00100000:3.0:1787828705.482219:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828705/real 1787828705] req@ffff9a78841da580 x1874671476861952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828705.482222:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828705.482233:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828705.482240:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828705.482241:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476861952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828705.482254:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828705.482258:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828705.482263:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476861952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828705.482265:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476862336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828705.482277:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828705.482280:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828705.482283:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828705.482284:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787828705.482290:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3072 00000100:00080000:2.0:1787828705.482296:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828705.482302:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828705.482306:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828705.482311:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828705.482319:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828705/real 1787828705] req@ffff9a78841d8000 x1874671476862080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828705.482338:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476862464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828705.482344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476862080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828705.482361:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828705.482364:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828705.482367:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828705.482372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828705.482385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828705.482392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828705/real 1787828705] req@ffff9a78841d83c0 x1874671476862208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828705.482399:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476862208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828705.482409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828705.482411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828705.482413:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828705.482416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828705.482426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828705.482431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828705/real 1787828705] req@ffff9a78841db480 x1874671476862336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828705.482438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476862336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828705.482445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828705.482446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828705.482448:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828705.482457:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828705.482494:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828705.482502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828705/real 1787828705] req@ffff9a78841d9e00 x1874671476862464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828705.482510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476862464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828705.482517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828705.482519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828705.482522:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828706.506008:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828707.529971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828708.554011:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828709.577961:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828710.601957:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828710.601969:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828710.601973:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828710.601983:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3078 00000100:00080000:2.0:1787828710.601996:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828710.602004:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787828710.602044:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828710.602046:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476862592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828710.602081:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828710.602085:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828710.602087:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828710.602092:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3078 00000100:00080000:2.0:1787828710.602096:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828710.602103:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828710.602113:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828710.602134:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476862720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828710.602134:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476862592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828710.602195:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828710.602202:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828710.602207:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828710.602223:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3078 00000100:00080000:2.0:1787828710.602253:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828710.602263:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828710.602268:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828710.602283:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828710.602367:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828710/real 1787828710] req@ffff9a78841d8b40 x1874671476862592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828710.602368:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476862848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828710.602421:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828710.602426:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828710.602429:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828710.602436:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3078 00000100:00100000:3.0:1787828710.602441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476862592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828710.602444:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828710.602459:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828710.602464:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:2.0:1787828710.602468:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787828710.602474:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828710.602481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476862592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828710.602490:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476862976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828710.602503:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828710.602506:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828710.602509:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828710.602515:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3078 00000100:00100000:3.0:1787828710.602522:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862720:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787828710.602525:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828710.602533:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828710.602539:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828710.602552:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476863104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828710.602555:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828710.602581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828710/real 1787828710] req@ffff9a78841db840 x1874671476862720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828710.602594:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476862720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828710.602602:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828710.602608:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828710.602611:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828710.602618:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828710.602631:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828710.602640:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828710/real 1787828710] req@ffff9a78841d92c0 x1874671476862848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828710.602647:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476862848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828710.602653:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828710.602655:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828710.602657:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828710.602661:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828710.602673:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828710.602679:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828710/real 1787828710] req@ffff9a78841d9680 x1874671476862976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828710.602687:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476862976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828710.602694:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828710.602695:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828710.602697:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476862976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828710.602708:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828710.602718:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828710.602724:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828710/real 1787828710] req@ffff9a78841da940 x1874671476863104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828710.602742:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476863104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828710.602750:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828710.602753:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828710.602755:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828711.626041:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828712.650043:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828713.674059:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828714.697983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828715.721963:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828715.721978:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828715.721984:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828715.721998:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3083 00000100:00080000:2.0:1787828715.722012:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828715.722023:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828715.722039:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828715.722076:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476863232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828715.722108:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828715.722112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828715.722115:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828715.722121:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3083 00000100:00080000:2.0:1787828715.722125:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828715.722131:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828715.722143:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476863232:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828715.722150:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828715.722170:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476863360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828715.722183:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828715.722187:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828715.722190:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828715.722195:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3083 00000100:00080000:2.0:1787828715.722208:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828715.722213:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828715.722217:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828715.722222:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828715.722240:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476863488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828715.722253:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828715.722270:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828715.722272:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828715.722278:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3083 00000100:00080000:2.0:1787828715.722283:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828715.722288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828715/real 1787828715] req@ffff9a78841da940 x1874671476863232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828715.722292:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828715.722304:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828715.722314:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476863232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828715.722321:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476863616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828715.722326:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828715.722335:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828715.722343:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476863232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828715.722344:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828715.722350:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828715.722353:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828715.722359:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3083 00000100:00080000:2.0:1787828715.722363:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828715.722369:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828715.722370:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863360:192.168.203.140@tcp:38:kworker.0 00000008:00100000:2.0:1787828715.722379:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787828715.722397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828715.722399:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476863744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828715.722411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828715/real 1787828715] req@ffff9a78841d9680 x1874671476863360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828715.722424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476863360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828715.722430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828715.722434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828715.722437:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828715.722445:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828715.722465:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828715.722478:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828715/real 1787828715] req@ffff9a78841d92c0 x1874671476863488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828715.722492:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476863488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828715.722500:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828715.722503:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828715.722505:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828715.722509:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828715.722526:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828715.722533:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828715/real 1787828715] req@ffff9a78841db840 x1874671476863616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828715.722541:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476863616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828715.722548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828715.722550:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828715.722552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828715.722563:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828715.722573:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828715.722579:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828715/real 1787828715] req@ffff9a78841d8b40 x1874671476863744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828715.722587:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476863744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828715.722594:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828715.722595:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828715.722597:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476863744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828716.745995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828717.770050:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828718.794052:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828719.818029:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828720.841971:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828720.841983:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828720.841988:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828720.842010:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3088 00000100:00080000:2.0:1787828720.842023:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828720.842031:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828720.842049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828720.842089:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476863872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828720.842114:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828720.842119:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828720.842121:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828720.842125:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3088 00000100:00080000:2.0:1787828720.842139:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828720.842142:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828720.842152:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828720.842161:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476864000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828720.842169:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476863872:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828720.842171:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828720.842178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828720.842181:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828720.842186:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3088 00000100:00080000:2.0:1787828720.842189:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828720.842191:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828720.842194:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828720.842202:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476864128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828720.842209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828720.842212:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828720.842213:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828720.842216:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3088 00000100:00080000:2.0:1787828720.842230:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828720.842235:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828720.842241:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828720.842250:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476864256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828720.842258:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828720.842260:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828720.842262:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828720.842264:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3088 00000100:00080000:2.0:1787828720.842267:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828720.842270:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828720.842273:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828720.842282:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476864384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828720.842327:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828720.842355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828720.842370:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828720.842386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828721.866106:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828722.889986:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828723.913998:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828723.914182:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787828723.914265:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828723.914282:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828723.914326:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828723.914333:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828723.914342:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828723.914355:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787828723.914397:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828720/real 1787828723] req@ffff9a78841db480 x1874671476864000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828723.914434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476864000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828723.914447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828723.914451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828723.914456:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828723.914470:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828720/real 1787828723] req@ffff9a78841d83c0 x1874671476864128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828723.914491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476864128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828723.914500:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828723.914504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828723.914507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828723.914515:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828720/real 1787828723] req@ffff9a78841d8000 x1874671476864256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828723.914529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476864256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828723.914540:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828723.914544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828723.914548:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828723.914556:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828720/real 1787828723] req@ffff9a78841da580 x1874671476864384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828723.914569:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476864384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828723.914580:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828723.914584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828723.914587:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864384:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828723.914594:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828720/real 1787828723] req@ffff9a78841d9e00 x1874671476863872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828723.914607:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476863872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828723.914615:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828723.914619:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828723.914622:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476863872:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787828724.938002:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828725.961945:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828725.961963:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828725.961968:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828725.961974:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828725.961980:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3093 00000100:00080000:2.0:1787828725.961995:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828725.962003:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828725.962048:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476864512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828725.962080:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828725.962084:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828725.962086:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828725.962091:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3093 00000100:00080000:2.0:1787828725.962096:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828725.962102:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828725.962115:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828725.962130:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476864512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828725.962136:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476864640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828725.962157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828725.962161:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828725.962163:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828725.962168:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3093 00000100:00080000:2.0:1787828725.962171:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828725.962202:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828725.962207:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828725.962221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476864768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828725.962229:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828725.962232:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828725.962233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828725.962237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3093 00000100:00080000:2.0:1787828725.962244:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828725.962248:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828725.962253:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828725.962262:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828725.962264:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476864896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828725.962282:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828725.962286:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828725.962288:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828725.962291:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3093 00000100:00080000:2.0:1787828725.962297:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828725.962302:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828725.962307:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828725.962331:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476865024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828725.962428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828725/real 1787828725] req@ffff9a78841d8780 x1874671476864512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828725.962457:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476864512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828725.962472:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828725.962477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828725.962481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476864512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828725.962517:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828725.962556:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828725.962570:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828725/real 1787828725] req@ffff9a78841d9a40 x1874671476864640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828725.962583:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476864640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828725.962598:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828725.962602:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828725.962605:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828725.962617:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828725.962637:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828725.962651:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828725/real 1787828725] req@ffff9a78841da1c0 x1874671476864768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828725.962662:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476864768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828725.962669:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828725.962673:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828725.962677:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828725.962691:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828725.962711:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828725.962722:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828725/real 1787828725] req@ffff9a78841db0c0 x1874671476864896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828725.962734:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476864896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828725.962744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828725.962750:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828725.962754:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476864896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828725.962762:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828725.962783:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828725.962795:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828725/real 1787828725] req@ffff9a78841dbc00 x1874671476865024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828725.962806:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476865024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828725.962814:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828725.962817:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828725.962821:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828726.986033:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828728.010076:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828729.033997:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828730.057968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828731.081935:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828731.081943:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828731.081947:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828731.081956:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3098 00000100:00080000:0.0:1787828731.081965:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828731.081970:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828731.081995:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828731.082003:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476865152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828731.082024:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828731.082027:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828731.082028:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828731.082031:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3098 00000100:00080000:0.0:1787828731.082033:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828731.082036:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828731.082044:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828731.082051:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476865152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828731.082057:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476865280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828731.082066:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828731.082068:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828731.082069:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828731.082072:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3098 00000100:00080000:0.0:1787828731.082074:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828731.082077:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828731.082081:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828731.082089:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476865408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828731.082095:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828731.082098:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828731.082099:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828731.082101:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3098 00000100:00080000:0.0:1787828731.082104:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828731.082108:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828731.082111:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828731.082117:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828731.082125:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476865536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828731.082132:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828731.082134:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828731.082135:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828731.082138:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3098 00000100:00080000:0.0:1787828731.082141:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828731.082144:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828731.082159:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828731.082164:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9a7890a3de00 x1874671476865152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828731.082172:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476865664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828731.082182:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476865152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828731.082191:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828731.082194:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828731.082197:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476865152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828731.082215:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828731.082232:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828731.082241:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9a7890a3da40 x1874671476865280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828731.082249:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476865280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828731.082254:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828731.082256:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828731.082259:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828731.082262:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828731.082272:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828731.082279:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9a7890a3c3c0 x1874671476865408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828731.082286:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476865408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828731.082291:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828731.082293:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828731.082295:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828731.082298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828731.082308:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828731.082314:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9a7890a3f480 x1874671476865536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828731.082321:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476865536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828731.082327:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828731.082330:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828731.082332:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828731.082335:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828731.082344:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828731.082356:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9a7890a3fc00 x1874671476865664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828731.082363:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476865664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828731.082370:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828731.082372:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828731.082374:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828732.105946:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828733.130010:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828734.154066:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828735.178044:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828736.202067:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828736.202096:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828736.202100:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828736.202136:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3103 00010000:00010000:1.0:1787828736.202146:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828736.202148:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828736.202174:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828736.202221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476865792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828736.202273:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828736.202279:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828736.202283:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828736.202291:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3103 00000100:00080000:2.0:1787828736.202296:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828736.202301:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828736.202331:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828736.202334:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476865792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828736.202354:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476865920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828736.202370:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828736.202374:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828736.202378:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828736.202385:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3103 00000100:00080000:2.0:1787828736.202390:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828736.202394:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828736.202401:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828736.202412:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476866048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828736.202423:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787828736.202432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828736.202438:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828736.202441:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828736.202447:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3103 00000100:00080000:2.0:1787828736.202453:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828736.202458:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828736.202465:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828736.202484:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476866176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828736.202493:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828736.202497:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828736.202501:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828736.202510:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3103 00000100:00080000:2.0:1787828736.202516:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828736.202520:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff9a78841dbc00 x1874671476865792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828736.202526:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828736.202557:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828736.202565:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476865792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828736.202573:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476866304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828736.202586:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828736.202596:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828736.202604:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476865792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828736.202646:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828736.202665:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828736.202674:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff9a78841db0c0 x1874671476865920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828736.202693:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476865920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828736.202700:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828736.202702:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828736.202705:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476865920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828736.202709:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828736.202719:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828736.202725:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff9a78841da1c0 x1874671476866048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828736.202733:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476866048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828736.202739:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828736.202741:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828736.202743:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828736.202747:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828736.202759:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828736.202765:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff9a78841d9a40 x1874671476866176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828736.202773:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476866176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828736.202780:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828736.202781:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828736.202783:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828736.202798:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828736.202807:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828736.202813:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828736/real 1787828736] req@ffff9a78841d8780 x1874671476866304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828736.202821:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476866304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828736.202827:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828736.202849:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828736.202851:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828737.226002:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828738.250023:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828739.274006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828740.298230:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828741.322153:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828741.322156:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.322173:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828741.322197:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828741.322209:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3108 00000100:00080000:0.0:1787828741.322222:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828741.322231:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828741.322292:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476866432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828741.322325:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.322354:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828741.322356:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828741.322362:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3108 00000100:00080000:0.0:1787828741.322366:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828741.322371:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828741.322382:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828741.322384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476866432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828741.322401:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476866560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828741.322418:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.322422:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828741.322424:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828741.322428:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3108 00000100:00080000:0.0:1787828741.322431:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828741.322435:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828741.322441:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828741.322452:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476866688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828741.322462:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.322465:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828741.322468:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828741.322472:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3108 00000100:00080000:0.0:1787828741.322479:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828741.322479:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828741.322492:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828741.322501:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828741.322521:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476866816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828741.322536:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828741.322541:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828741.322544:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828741.322551:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828741] req@ffff9a7890a3c000 x1874671476866432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828741.322557:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3108 00000100:00080000:0.0:1787828741.322589:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828741.322597:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828741.322606:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476866432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828741.322611:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828741.322641:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787828741.322647:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476866944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828741.322652:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828741.322687:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476866432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828741.322732:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828741.322757:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828741.322797:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828741] req@ffff9a7890a3fc00 x1874671476866560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828741.322811:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476866560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828741.322819:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828741.322822:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828741.322826:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828741.322832:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828741.322849:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828741.322907:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828741] req@ffff9a7890a3f480 x1874671476866688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828741.322917:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476866688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828741.322935:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828741.322938:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828741.322941:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828741.322946:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828741.322963:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828741.322970:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828741] req@ffff9a7890a3c3c0 x1874671476866816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828741.322978:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476866816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828741.322986:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828741.322987:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828741.322989:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828741.322994:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828741.323005:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828741.323011:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828741/real 1787828741] req@ffff9a7890a3da40 x1874671476866944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828741.323019:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476866944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828741.323026:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828741.323028:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828741.323030:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476866944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828742.346053:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828743.370101:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828744.394043:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828745.418002:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828746.442030:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828746.442064:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828746.442071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828746.442084:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3113 00000100:00080000:2.0:1787828746.442097:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828746.442107:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828746.442151:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828746.442157:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476867072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828746.442199:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828746.442204:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828746.442207:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828746.442211:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3113 00000100:00080000:2.0:1787828746.442215:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828746.442220:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828746.442233:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828746.442260:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476867200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828746.442285:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476867072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828746.442307:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828746.442316:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828746.442321:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828746.442331:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3113 00000100:00080000:2.0:1787828746.442339:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828746.442346:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828746.442365:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828746.442386:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476867328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828746.442395:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828746.442413:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828746.442418:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828746.442422:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828746.442430:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3113 00000100:00080000:2.0:1787828746.442434:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828746.442439:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828746.442446:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828746.442462:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476867456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828746.442476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff9a78841d9e00 x1874671476867072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828746.442481:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828746.442498:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828746.442505:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828746.442530:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3113 00000100:00080000:2.0:1787828746.442535:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828746.442543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476867072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828746.442550:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828746.442569:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828746.442579:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828746.442593:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828746.442600:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476867584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828746.442610:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476867072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828746.442670:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828746.442707:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828746.442725:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff9a78841da580 x1874671476867200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828746.442742:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476867200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828746.442778:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828746.442783:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828746.442792:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828746.442800:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828746.442823:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828746.442876:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff9a78841d8000 x1874671476867328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828746.442903:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476867328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828746.442927:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828746.442931:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828746.442935:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828746.442942:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828746.442976:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828746.442988:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff9a78841d83c0 x1874671476867456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828746.443002:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476867456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828746.443014:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828746.443017:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828746.443021:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828746.443028:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828746.443085:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828746.443106:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828746/real 1787828746] req@ffff9a78841db480 x1874671476867584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828746.443122:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476867584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828746.443134:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828746.443138:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828746.443151:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828747.466006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828748.490054:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828749.514084:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828750.538033:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828751.561922:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828751.561932:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828751.561937:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828751.561946:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3119 00000100:00080000:0.0:1787828751.561956:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828751.561956:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828751.561965:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828751.561994:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476867712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828751.562027:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828751.562030:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828751.562032:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828751.562035:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3119 00000100:00080000:0.0:1787828751.562038:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828751.562042:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828751.562054:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828751.562060:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476867712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828751.562069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476867840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828751.562078:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828751.562081:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828751.562082:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828751.562086:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3119 00000100:00080000:0.0:1787828751.562088:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828751.562090:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828751.562093:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828751.562101:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476867968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828751.562112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828751.562116:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828751.562119:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828751.562120:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828751.562124:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3119 00000100:00080000:0.0:1787828751.562128:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828751.562130:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828751.562134:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828751.562146:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476868096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828751.562156:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828751.562160:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828751.562162:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828751.562168:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3119 00000100:00100000:3.0:1787828751.562172:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828751/real 1787828751] req@ffff9a7890a3de00 x1874671476867712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828751.562174:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828751.562182:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828751.562192:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828751.562203:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476868224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828751.562215:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476867712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828751.562227:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828751.562230:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828751.562233:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476867712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828751.562252:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828751.562272:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828751.562282:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828751/real 1787828751] req@ffff9a7890a3da40 x1874671476867840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828751.562290:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476867840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828751.562296:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828751.562298:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828751.562301:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828751.562305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828751.562315:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828751.562322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828751/real 1787828751] req@ffff9a7890a3c3c0 x1874671476867968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828751.562330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476867968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828751.562335:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828751.562337:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828751.562339:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476867968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828751.562343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828751.562352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828751.562361:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828751/real 1787828751] req@ffff9a7890a3f480 x1874671476868096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828751.562373:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476868096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828751.562384:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828751.562387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828751.562391:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828751.562398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828751.562418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828751.562427:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828751/real 1787828751] req@ffff9a7890a3fc00 x1874671476868224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828751.562435:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476868224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828751.562442:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828751.562444:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828751.562446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828752.585965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828753.609965:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828754.633957:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828755.657938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828756.681905:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828756.681917:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828756.681922:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828756.681932:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3124 00000100:00080000:0.0:1787828756.681944:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828756.681952:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828756.681963:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828756.682023:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476868352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828756.682067:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828756.682074:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828756.682077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828756.682086:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3124 00000100:00080000:0.0:1787828756.682094:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828756.682102:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828756.682113:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476868352:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787828756.682120:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828756.682153:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476868480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828756.682176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828756.682183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828756.682187:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828756.682192:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828756.682219:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3124 00000100:00080000:0.0:1787828756.682229:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828756.682237:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828756.682248:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828756.682248:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828756/real 1787828756] req@ffff9a7890a3c000 x1874671476868352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828756.682288:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476868608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828756.682289:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476868352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828756.682367:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828756.682371:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828756.682378:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787828756.682383:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1787828756.682391:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476868352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828756.682392:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828756.682408:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3124 00000100:00080000:0.0:1787828756.682415:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828756.682423:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828756.682445:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868480:192.168.203.140@tcp:38:kworker.0 00000008:00100000:0.0:1787828756.682447:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828756.682465:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476868736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828756.682473:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828756.682509:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828756.682514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828756/real 1787828756] req@ffff9a7890a3fc00 x1874671476868480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828756.682519:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828756.682527:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828756.682543:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3124 00000100:00100000:3.0:1787828756.682544:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476868480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828756.682553:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828756.682562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828756.682569:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828756.682571:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828756.682581:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868480:192.168.203.140@tcp:38:kworker.0 00000008:00100000:0.0:1787828756.682602:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828756.682614:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828756.682623:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476868864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828756.682634:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828756.682647:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828756/real 1787828756] req@ffff9a7890a3f480 x1874671476868608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828756.682657:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476868608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828756.682663:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828756.682665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828756.682667:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828756.682679:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828756.682697:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828756.682704:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828756/real 1787828756] req@ffff9a7890a3c3c0 x1874671476868736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828756.682711:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476868736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828756.682718:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828756.682721:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828756.682723:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828756.682726:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828756.682736:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828756.682742:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828756/real 1787828756] req@ffff9a7890a3da40 x1874671476868864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828756.682749:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476868864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828756.682755:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828756.682757:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828756.682759:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476868864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828757.705975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828758.729993:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828759.753976:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828760.778044:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828761.801919:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828761.801930:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828761.801949:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828761.801962:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3129 00000100:00080000:0.0:1787828761.801974:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828761.801979:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828761.801982:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828761.802024:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476868992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828761.802048:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828761.802051:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828761.802053:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828761.802056:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3129 00000100:00080000:0.0:1787828761.802059:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828761.802063:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828761.802073:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828761.802085:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476868992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828761.802087:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476869120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828761.802099:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828761.802102:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828761.802104:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828761.802107:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3129 00000100:00080000:0.0:1787828761.802109:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828761.802112:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828761.802115:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828761.802125:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476869248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828761.802133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828761.802136:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828761.802137:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828761.802140:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3129 00000100:00080000:0.0:1787828761.802144:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828761.802148:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828761.802152:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828761.802158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828761.802168:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476869376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828761.802175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828761.802178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828761.802180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828761.802183:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3129 00000100:00080000:0.0:1787828761.802187:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828761.802190:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828761.802193:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828761.802206:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828761/real 1787828761] req@ffff9a7890a3de00 x1874671476868992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828761.802208:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476869504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828761.802267:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476868992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828761.802278:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828761.802282:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828761.802285:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476868992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828761.802305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828761.802321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828761.802330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828761/real 1787828761] req@ffff9a7890a3da40 x1874671476869120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828761.802339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476869120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828761.802349:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828761.802352:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828761.802354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828761.802358:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828761.802368:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828761.802374:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828761/real 1787828761] req@ffff9a7890a3c3c0 x1874671476869248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828761.802381:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476869248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828761.802387:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828761.802389:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828761.802391:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828761.802394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828761.802405:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828761.802410:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828761/real 1787828761] req@ffff9a7890a3f480 x1874671476869376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828761.802418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476869376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828761.802424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828761.802426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828761.802429:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828761.802433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828761.802442:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828761.802448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828761/real 1787828761] req@ffff9a7890a3fc00 x1874671476869504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828761.802455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476869504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828761.802462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828761.802463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828761.802465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828762.826052:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828763.850082:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828764.874070:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828765.897942:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828766.921929:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828766.921957:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828766.921961:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828766.921970:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828766.921972:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3134 00000100:00080000:0.0:1787828766.921984:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828766.921992:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828766.922053:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476869632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828766.922084:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828766.922092:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828766.922095:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828766.922101:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3134 00000100:00080000:0.0:1787828766.922106:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828766.922113:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476869632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828766.922114:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828766.922129:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828766.922157:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476869760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828766.922170:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828766.922174:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828766.922176:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828766.922181:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3134 00000800:00000100:3.0:1787828766.922183:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828766.922189:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828766.922197:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828766.922203:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828766.922244:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476869888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828766.922255:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828766.922256:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828766/real 1787828766] req@ffff9a7890a3c000 x1874671476869632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828766.922266:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828766.922270:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828766.922277:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3134 00000100:00100000:3.0:1787828766.922278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476869632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828766.922285:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828766.922290:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828766.922294:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787828766.922296:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828766.922301:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476869632:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787828766.922306:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828766.922321:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476870016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828766.922323:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869760:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787828766.922335:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828766.922339:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828766.922342:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828766.922348:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3134 00000100:00080000:0.0:1787828766.922353:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828766.922356:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828766.922361:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828766.922368:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828766.922376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828766/real 1787828766] req@ffff9a7890a3fc00 x1874671476869760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828766.922382:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476870144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828766.922402:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476869760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828766.922410:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828766.922413:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828766.922416:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828766.922421:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828766.922436:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828766.922444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828766/real 1787828766] req@ffff9a7890a3f480 x1874671476869888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828766.922451:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476869888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828766.922456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828766.922458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828766.922460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476869888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828766.922470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828766.922503:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828766.922514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828766/real 1787828766] req@ffff9a7890a3c3c0 x1874671476870016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828766.922527:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476870016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828766.922539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828766.922541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828766.922544:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828766.922548:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828766.922578:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828766.922585:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828766/real 1787828766] req@ffff9a7890a3da40 x1874671476870144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828766.922593:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476870144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828766.922599:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828766.922601:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828766.922603:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828767.945989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828768.970048:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828769.994010:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828771.017970:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828772.041967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828772.041974:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.041981:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828772.041986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.041994:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3139 00000100:00080000:3.0:1787828772.042003:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828772.042008:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828772.042038:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476870272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828772.042060:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.042063:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828772.042065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.042067:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3139 00000100:00080000:3.0:1787828772.042070:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828772.042073:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828772.042081:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828772.042090:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476870400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828772.042099:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.042101:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828772.042102:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.042108:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3139 00000100:00080000:3.0:1787828772.042114:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828772.042117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476870272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828772.042118:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828772.042129:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828772.042139:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476870528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828772.042148:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.042151:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828772.042153:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.042156:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3139 00000100:00080000:3.0:1787828772.042158:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828772.042160:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828772.042164:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828772.042176:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476870656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828772.042182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828772.042185:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828772.042186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828772.042189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3139 00000100:00080000:3.0:1787828772.042203:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828772.042206:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828772.042209:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:0.0:1787828772.042215:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828772.042220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476870784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828772.042274:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff9a78a6777480 x1874671476870272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828772.042290:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476870272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828772.042300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828772.042303:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828772.042307:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476870272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828772.042331:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828772.042348:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828772.042357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff9a78a6774000 x1874671476870400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828772.042366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476870400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828772.042372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828772.042383:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828772.042386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828772.042393:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787828772.042415:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828772.042427:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff9a78a6775a40 x1874671476870528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828772.042440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476870528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828772.042447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828772.042450:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828772.042452:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828772.042457:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828772.042473:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828772.042480:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff9a78a67752c0 x1874671476870656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828772.042493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476870656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828772.042500:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828772.042502:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828772.042503:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787828772.042507:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787828772.042517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828772.042523:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828772/real 1787828772] req@ffff9a78a6774b40 x1874671476870784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828772.042531:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476870784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828772.042537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828772.042539:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828772.042541:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476870784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828773.065975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828774.089986:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828775.113997:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828776.138003:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828777.161995:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828777.162007:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828777.162011:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828777.162022:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3144 00000100:00080000:0.0:1787828777.162034:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828777.162041:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828777.162070:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828777.162087:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476870912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828777.162115:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828777.162118:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828777.162119:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828777.162122:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3144 00000100:00080000:0.0:1787828777.162125:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828777.162129:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828777.162139:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828777.162150:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476870912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828777.162153:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476871040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828777.162163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828777.162165:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828777.162166:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828777.162169:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3144 00000100:00080000:0.0:1787828777.162171:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828777.162174:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828777.162177:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828777.162185:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476871168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828777.162191:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828777.162193:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828777.162194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828777.162197:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3144 00000100:00080000:0.0:1787828777.162214:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828777.162217:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828777.162222:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828777.162226:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828777.162233:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476871296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828777.162242:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828777.162245:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828777.162246:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828777.162249:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3144 00000100:00080000:0.0:1787828777.162251:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828777.162253:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828777.162256:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828777.162265:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476871424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828777.162291:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff9a7890a3de00 x1874671476870912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828777.162318:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476870912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828777.162327:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828777.162331:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828777.162334:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476870912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828777.162356:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828777.162372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828777.162380:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff9a7890a3da40 x1874671476871040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828777.162388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476871040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828777.162394:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828777.162396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828777.162398:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828777.162402:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828777.162412:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828777.162418:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff9a7890a3c3c0 x1874671476871168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828777.162425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476871168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828777.162430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828777.162432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828777.162434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828777.162437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828777.162447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828777.162453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff9a7890a3f480 x1874671476871296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828777.162465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476871296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828777.162472:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828777.162473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828777.162475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828777.162478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828777.162488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828777.162494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828777/real 1787828777] req@ffff9a7890a3fc00 x1874671476871424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828777.162501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476871424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828777.162508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828777.162510:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828777.162511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828778.186008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828779.210006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828780.234068:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828781.258022:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828782.281962:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828782.281974:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:1.0:1787828782.281978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828782.281980:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828782.281993:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3149 00000100:00080000:2.0:1787828782.282006:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828782.282013:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828782.282057:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476871552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828782.282082:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828782.282086:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828782.282087:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828782.282091:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3149 00000100:00080000:2.0:1787828782.282095:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828782.282099:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828782.282109:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828782.282121:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476871680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828782.282126:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476871552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828782.282150:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828782.282153:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828782.282154:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828782.282160:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3149 00000100:00080000:2.0:1787828782.282162:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828782.282165:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828782.282168:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828782.282183:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476871808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828782.282190:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828782.282193:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828782.282194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828782.282197:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3149 00000100:00080000:2.0:1787828782.282207:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828782.282210:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828782.282215:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828782.282216:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828782.282235:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476871936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828782.282243:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828782.282246:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828782.282248:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828782.282252:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3149 00000100:00080000:2.0:1787828782.282255:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828782.282258:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828782.282263:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828782.282277:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476872064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828782.282285:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff9a78841d8b40 x1874671476871552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828782.282308:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476871552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828782.282322:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828782.282325:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828782.282328:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476871552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828782.282350:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828782.282370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828782.282379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff9a78841db840 x1874671476871680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828782.282387:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476871680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828782.282393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828782.282396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828782.282398:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828782.282402:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828782.282412:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828782.282426:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff9a78841d92c0 x1874671476871808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828782.282434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476871808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828782.282440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828782.282442:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828782.282444:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828782.282449:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828782.282460:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828782.282466:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff9a78841d9680 x1874671476871936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828782.282473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476871936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828782.282480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828782.282487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828782.282489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476871936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828782.282493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828782.282504:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828782.282510:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828782/real 1787828782] req@ffff9a78841da940 x1874671476872064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828782.282519:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476872064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828782.282528:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828782.282537:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828782.282542:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828783.306022:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828784.329988:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828785.354037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828786.378041:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828787.401923:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828787.401932:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828787.401937:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828787.401947:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3154 00000100:00080000:0.0:1787828787.401958:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828787.401983:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828787.401984:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828787.402020:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476872192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828787.402061:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828787.402066:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828787.402067:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828787.402070:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3154 00000100:00080000:0.0:1787828787.402073:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828787.402076:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828787.402085:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828787.402112:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476872320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828787.402118:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476872192:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828787.402135:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828787.402138:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828787.402140:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828787.402144:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3154 00000100:00080000:0.0:1787828787.402147:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828787.402150:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828787.402154:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828787.402166:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476872448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828787.402175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828787.402178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828787.402197:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828787.402200:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3154 00000100:00080000:0.0:1787828787.402204:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828787.402207:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828787.402212:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828787.402221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476872576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828787.402229:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828787.402232:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828787.402234:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828787.402237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3154 00000100:00080000:0.0:1787828787.402240:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828787.402243:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828787.402248:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828787.402257:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476872704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828787.402284:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828787.402351:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828787.402375:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828787.402395:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828788.425981:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828789.449949:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828790.474006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828790.474105:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787828790.474150:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828790.474163:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828790.474197:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828790.474207:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828790.474220:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787828790.474232:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787828790.474245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828790] req@ffff9a7890a3fc00 x1874671476872320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828790.474269:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476872320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828790.474281:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828790.474286:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828790.474300:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828790.474308:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828790] req@ffff9a7890a3f480 x1874671476872448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828790.474320:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476872448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828790.474328:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828790.474332:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828790.474335:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828790.474353:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828790] req@ffff9a7890a3c3c0 x1874671476872576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828790.474367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476872576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828790.474378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828790.474382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828790.474385:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828790.474391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828790] req@ffff9a7890a3c000 x1874671476872192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828790.474408:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476872192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828790.474415:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828790.474418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828790.474421:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476872192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828790.474446:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828787/real 1787828790] req@ffff9a7890a3da40 x1874671476872704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828790.474458:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476872704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828790.474468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828790.474471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828790.474474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828791.497977:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828792.521936:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828792.521949:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828792.521955:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828792.521968:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3160 00000100:00080000:0.0:1787828792.521982:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828792.521991:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828792.522002:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828792.522040:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476872832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828792.522067:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828792.522071:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828792.522072:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828792.522075:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3160 00000100:00080000:0.0:1787828792.522079:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828792.522083:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828792.522094:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828792.522108:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476872960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828792.522115:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828792.522118:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828792.522120:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828792.522126:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476872832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828792.522128:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3160 00000100:00080000:0.0:1787828792.522135:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828792.522142:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828792.522147:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828792.522163:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476873088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828792.522174:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828792.522178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828792.522180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828792.522185:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3160 00000100:00080000:0.0:1787828792.522208:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828792.522214:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828792.522219:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828792.522224:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828792.522237:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476873216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828792.522254:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828792.522258:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828792.522260:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828792.522265:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3160 00000100:00080000:0.0:1787828792.522275:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828792.522281:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828792.522286:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828792.522295:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828792/real 1787828792] req@ffff9a7890a3de00 x1874671476872832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828792.522304:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476873344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828792.522333:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476872832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828792.522348:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828792.522353:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828792.522358:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476872832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828792.522394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828792.522420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828792.522435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828792/real 1787828792] req@ffff9a7890a3da40 x1874671476872960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828792.522450:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476872960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828792.522459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828792.522463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828792.522466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476872960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828792.522473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828792.522495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828792.522508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828792/real 1787828792] req@ffff9a7890a3c000 x1874671476873088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828792.522522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476873088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828792.522538:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828792.522542:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828792.522545:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828792.522551:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828792.522573:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828792.522586:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828792/real 1787828792] req@ffff9a7890a3c3c0 x1874671476873216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828792.522599:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476873216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828792.522610:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828792.522614:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828792.522617:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828792.522623:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828792.522644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828792.522658:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828792/real 1787828792] req@ffff9a7890a3f480 x1874671476873344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828792.522670:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476873344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828792.522681:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828792.522684:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828792.522688:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828793.545979:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828794.570006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828795.594045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828796.618027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828797.641951:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828797.641961:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828797.641965:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828797.641978:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3165 00000100:00080000:0.0:1787828797.641990:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828797.641997:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828797.642043:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476873472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828797.642064:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828797.642067:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828797.642071:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828797.642073:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828797.642077:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3165 00000100:00080000:0.0:1787828797.642080:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828797.642083:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828797.642094:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828797.642106:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476873600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828797.642115:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828797.642117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476873472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828797.642118:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828797.642122:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828797.642129:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3165 00000100:00080000:0.0:1787828797.642132:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828797.642134:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828797.642137:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828797.642146:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476873728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828797.642152:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828797.642155:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828797.642156:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828797.642159:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3165 00000100:00080000:0.0:1787828797.642161:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828797.642163:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828797.642166:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828797.642173:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476873856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828797.642179:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828797.642181:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828797.642182:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828797.642185:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3165 00000100:00080000:0.0:1787828797.642195:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828797.642198:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828797.642202:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787828797.642223:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828797.642230:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476873984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828797.642306:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828797/real 1787828797] req@ffff9a7890a3fc00 x1874671476873472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828797.642329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476873472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828797.642353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828797.642359:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828797.642363:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476873472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828797.642394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828797.642429:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828797.642444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828797/real 1787828797] req@ffff9a7890a3f480 x1874671476873600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828797.642458:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476873600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828797.642468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828797.642471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828797.642476:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828797.642482:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828797.642503:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828797.642516:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828797/real 1787828797] req@ffff9a7890a3c3c0 x1874671476873728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828797.642529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476873728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828797.642538:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828797.642541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828797.642545:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828797.642552:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828797.642572:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828797.642585:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828797/real 1787828797] req@ffff9a7890a3c000 x1874671476873856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828797.642598:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476873856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828797.642610:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828797.642613:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828797.642617:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828797.642624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828797.642643:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828797.642654:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828797/real 1787828797] req@ffff9a7890a3da40 x1874671476873984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828797.642666:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476873984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828797.642677:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828797.642680:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828797.642699:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476873984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828798.666050:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828799.690066:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828800.714017:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828801.738053:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828802.761983:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828802.761994:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828802.762015:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828802.762025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828802.762028:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3170 00000100:00080000:0.0:1787828802.762042:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828802.762056:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828802.762103:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476874112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828802.762143:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828802.762146:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828802.762147:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828802.762151:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3170 00000100:00080000:0.0:1787828802.762155:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828802.762158:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828802.762169:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828802.762177:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476874112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828802.762184:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476874240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828802.762214:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828802.762218:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828802.762220:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828802.762223:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3170 00000100:00080000:0.0:1787828802.762225:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828802.762229:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828802.762233:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828802.762244:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476874368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828802.762251:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828802.762255:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828802.762256:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828802.762259:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3170 00000100:00080000:0.0:1787828802.762261:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828802.762264:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828802.762267:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828802.762275:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828802.762279:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476874496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828802.762292:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828802.762297:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828802.762299:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828802.762305:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3170 00000100:00080000:0.0:1787828802.762310:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828802.762315:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828802.762323:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828802.762351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9a7890a3de00 x1874671476874112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828802.762357:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476874624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828802.762380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476874112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828802.762391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828802.762397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828802.762401:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476874112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828802.762430:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828802.762454:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828802.762464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9a7890a3da40 x1874671476874240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828802.762473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476874240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828802.762483:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828802.762486:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828802.762490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828802.762498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828802.762519:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828802.762530:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9a7890a3c000 x1874671476874368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828802.762539:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476874368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828802.762545:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828802.762548:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828802.762550:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828802.762554:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828802.762567:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828802.762573:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9a7890a3c3c0 x1874671476874496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828802.762581:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476874496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828802.762588:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828802.762590:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828802.762592:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828802.762596:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828802.762606:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828802.762611:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9a7890a3f480 x1874671476874624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828802.762619:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476874624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828802.762626:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828802.762628:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828802.762630:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828803.785975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828804.810095:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828805.834111:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828806.858117:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828807.881973:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828807.881999:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828807.882018:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828807.882039:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3175 00000100:00080000:0.0:1787828807.882058:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828807.882059:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828807.882071:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828807.882130:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476874752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828807.882165:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828807.882186:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828807.882188:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828807.882192:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3175 00000100:00080000:0.0:1787828807.882197:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828807.882202:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828807.882213:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828807.882228:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476874752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828807.882233:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476874880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828807.882246:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828807.882253:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828807.882254:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828807.882259:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3175 00000100:00080000:0.0:1787828807.882264:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828807.882267:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828807.882272:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828807.882295:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476875008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828807.882305:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828807.882309:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828807.882311:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828807.882314:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3175 00000100:00080000:0.0:1787828807.882329:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828807.882333:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828807.882337:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828807.882355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828807.882362:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476875136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828807.882374:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828807.882379:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828807.882382:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828807.882389:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3175 00000100:00080000:0.0:1787828807.882395:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828807.882401:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828807.882410:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828807.882430:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476875264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828807.882438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9a7890a3fc00 x1874671476874752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828807.882481:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476874752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828807.882497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828807.882502:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828807.882507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476874752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828807.882554:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828807.882586:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828807.882604:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9a7890a3f480 x1874671476874880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828807.882670:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476874880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828807.882685:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828807.882688:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828807.882692:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476874880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828807.882698:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828807.882737:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828807.882748:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9a7890a3c3c0 x1874671476875008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828807.882759:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476875008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828807.882767:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828807.882769:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828807.882772:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828807.882778:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828807.882793:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828807.882802:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9a7890a3c000 x1874671476875136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828807.882812:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476875136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828807.882821:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828807.882824:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828807.882828:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828807.882833:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828807.882893:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828807.882906:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9a7890a3da40 x1874671476875264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828807.882918:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476875264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828807.882929:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828807.882932:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828807.882935:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828808.905999:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828809.930086:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828810.954094:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828811.978007:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828813.001936:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828813.001944:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828813.001947:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828813.002002:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3180 00000100:00080000:0.0:1787828813.002011:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828813.002017:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828813.002028:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828813.002050:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476875392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828813.002071:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828813.002074:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828813.002075:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828813.002078:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3180 00000100:00080000:0.0:1787828813.002081:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828813.002083:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828813.002091:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828813.002101:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476875520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828813.002105:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476875392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828813.002110:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828813.002113:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828813.002114:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828813.002127:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3180 00000100:00080000:0.0:1787828813.002130:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828813.002133:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828813.002136:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828813.002145:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476875648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828813.002152:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828813.002155:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828813.002156:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828813.002159:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3180 00000100:00080000:0.0:1787828813.002161:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828813.002164:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828813.002168:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828813.002174:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476875776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828813.002180:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828813.002183:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828813.002184:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828813.002188:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3180 00000100:00080000:0.0:1787828813.002190:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787828813.002193:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828813.002196:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828813.002200:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828813.002209:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476875904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828813.002244:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9a7890a3de00 x1874671476875392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.002261:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476875392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.002269:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.002273:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828813.002276:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476875392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828813.002307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828813.002323:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828813.002332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9a7890a3da40 x1874671476875520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.002341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476875520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.002347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.002349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828813.002352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828813.002355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828813.002367:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828813.002378:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9a7890a3c000 x1874671476875648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.002390:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476875648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.002399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.002415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828813.002418:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828813.002425:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828813.002443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828813.002451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9a7890a3c3c0 x1874671476875776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.002459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476875776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.002466:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.002478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828813.002481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828813.002484:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828813.002494:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828813.002500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828813/real 1787828813] req@ffff9a7890a3f480 x1874671476875904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828813.002507:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476875904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828813.002513:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828813.002514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828813.002516:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476875904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828814.025980:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828815.050057:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828816.074031:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828817.098063:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828818.121899:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.121908:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828818.121912:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828818.121921:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3185 00000100:00080000:0.0:1787828818.121930:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828818.121936:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828818.121947:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828818.121966:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476876032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828818.121988:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.121992:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828818.121994:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828818.121998:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3185 00000100:00080000:0.0:1787828818.122002:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828818.122007:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828818.122019:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476876032:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787828818.122022:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828818.122042:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476876160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828818.122053:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.122056:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828818.122057:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828818.122060:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3185 00000100:00080000:0.0:1787828818.122064:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828818.122067:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828818.122071:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828818.122073:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828818.122084:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476876288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828818.122092:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.122094:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828818.122095:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828818.122097:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3185 00000100:00080000:0.0:1787828818.122100:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828818.122102:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828818.122105:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828818.122112:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476876416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828818.122118:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828818.122120:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828818.122122:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828818.122127:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3185 00000100:00080000:0.0:1787828818.122131:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828818.122136:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff9a7890a3fc00 x1874671476876032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828818.122137:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828818.122145:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828818.122164:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476876544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828818.122165:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476876032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.122196:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.122201:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828818.122206:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476876032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828818.122237:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828818.122258:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828818.122269:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff9a7890a3f480 x1874671476876160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828818.122279:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476876160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.122285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.122288:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828818.122291:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828818.122296:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828818.122306:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828818.122312:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff9a7890a3c3c0 x1874671476876288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828818.122320:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476876288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.122325:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.122327:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828818.122329:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828818.122332:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828818.122351:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828818.122374:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff9a7890a3c000 x1874671476876416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828818.122382:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476876416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.122389:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.122391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828818.122393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828818.122397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828818.122408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828818.122413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828818/real 1787828818] req@ffff9a7890a3da40 x1874671476876544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828818.122421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476876544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828818.122427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828818.122428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828818.122430:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828819.145955:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828820.169975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828821.194033:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828822.217976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828823.242065:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828823.242080:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828823.242087:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828823.242103:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3190 00000100:00080000:0.0:1787828823.242118:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828823.242128:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828823.242184:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476876672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828823.242231:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828823.242238:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828823.242242:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828823.242261:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3190 00000100:00080000:0.0:1787828823.242279:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828823.242286:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828823.242289:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828823.242306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476876672:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787828823.242309:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828823.242344:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476876800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828823.242360:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828823.242365:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828823.242367:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828823.242372:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3190 00000100:00080000:0.0:1787828823.242439:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828823.242444:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828823.242450:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787828823.242452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828823.242472:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476876928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828823.242493:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828823.242497:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828823.242499:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828823.242504:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3190 00000100:00080000:0.0:1787828823.242533:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828823.242541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff9a7890a3de00 x1874671476876672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828823.242544:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828823.242560:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828823.242583:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476876672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828823.242590:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476877056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828823.242641:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828823.242650:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828823.242653:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828823.242665:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476876672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828823.242668:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828823.242676:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828823.242685:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3190 00000100:00080000:0.0:1787828823.242693:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828823.242700:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828823.242709:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828823.242730:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828823.242738:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476877184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828823.242770:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828823.242791:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff9a7890a3da40 x1874671476876800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828823.242810:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476876800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828823.242821:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828823.242826:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828823.242859:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828823.242867:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828823.242897:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828823.242934:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff9a7890a3c000 x1874671476876928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828823.242967:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476876928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828823.242983:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828823.242988:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828823.242994:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476876928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828823.243004:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828823.243041:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828823.243061:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff9a7890a3c3c0 x1874671476877056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828823.243082:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476877056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828823.243098:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828823.243104:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828823.243110:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828823.243131:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828823.243156:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828823.243168:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828823/real 1787828823] req@ffff9a7890a3f480 x1874671476877184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828823.243181:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476877184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828823.243205:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828823.243211:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828823.243216:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828824.265979:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828825.289948:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828826.314000:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828827.337966:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828828.362027:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.362039:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828828.362044:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.362067:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3195 00000100:00080000:3.0:1787828828.362080:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828828.362086:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828828.362087:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828828.362198:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476877312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828828.362221:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.362225:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828828.362226:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.362229:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3195 00000100:00080000:3.0:1787828828.362232:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828828.362235:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828828.362244:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828828.362265:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476877440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828828.362273:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.362276:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828828.362278:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.362281:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3195 00000100:00080000:3.0:1787828828.362284:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828828.362288:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828828.362291:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828828.362300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476877312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828828.362301:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476877568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828828.362328:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.362332:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828828.362334:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.362339:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3195 00000100:00080000:3.0:1787828828.362343:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828828.362347:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828828.362353:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828828.362361:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476877696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828828.362368:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828828.362370:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828828.362389:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828828.362393:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3195 00000100:00080000:3.0:1787828828.362397:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828828.362400:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787828828.362406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787828828.362407:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828828.362423:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476877824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828828.362464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff9a78a6776580 x1874671476877312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828828.362498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476877312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828828.362508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828828.362522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828828.362526:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476877312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828828.362533:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828828.362555:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828828.362565:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff9a78a6776940 x1874671476877440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828828.362574:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476877440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828828.362580:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828828.362584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828828.362586:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828828.362610:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828828.362623:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828828.362630:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff9a78a67770c0 x1874671476877568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828828.362639:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476877568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828828.362645:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828828.362647:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828828.362665:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828828.362671:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828828.362686:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828828.362694:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff9a78a6774780 x1874671476877696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828828.362703:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476877696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828828.362711:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828828.362729:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828828.362732:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828828.362737:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828828.362751:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828828.362767:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828828/real 1787828828] req@ffff9a78a67761c0 x1874671476877824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828828.362777:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476877824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828828.362800:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828828.362804:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828828.362808:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476877824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828829.386003:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828830.410078:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828831.433985:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828832.458101:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828833.481954:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828833.481963:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828833.481967:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828833.481977:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3200 00000100:00080000:1.0:1787828833.481986:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828833.481993:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828833.482019:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787828833.482029:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476877952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828833.482059:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828833.482062:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828833.482064:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828833.482066:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3200 00000100:00080000:1.0:1787828833.482069:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828833.482072:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828833.482080:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828833.482090:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476878080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828833.482098:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828833.482103:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828833.482103:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476877952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828833.482106:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828833.482113:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3200 00000100:00080000:1.0:1787828833.482117:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828833.482120:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828833.482123:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828833.482133:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476878208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828833.482140:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828833.482143:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828833.482144:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828833.482147:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3200 00000100:00080000:1.0:1787828833.482150:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828833.482153:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828833.482157:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828833.482164:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476878336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828833.482172:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828833.482174:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828833.482176:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828833.482182:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3200 00000800:00000100:2.0:1787828833.482183:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828833.482186:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828833.482190:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828833.482204:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828833.482211:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476878464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828833.482267:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828833/real 1787828833] req@ffff9a7890b1f840 x1874671476877952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828833.482288:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476877952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828833.482312:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828833.482316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828833.482321:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476877952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828833.482346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828833.482378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828833.482391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828833/real 1787828833] req@ffff9a7890b1f480 x1874671476878080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828833.482407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476878080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828833.482417:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828833.482421:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828833.482426:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828833.482433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828833.482472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828833.482486:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828833/real 1787828833] req@ffff9a7890b1f0c0 x1874671476878208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828833.482501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476878208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828833.482511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828833.482514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828833.482518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828833.482526:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828833.482548:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828833.482559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828833/real 1787828833] req@ffff9a7890b1c000 x1874671476878336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828833.482573:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476878336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828833.482592:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828833.482596:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828833.482600:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828833.482607:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828833.482630:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828833.482644:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828833/real 1787828833] req@ffff9a7890b1fc00 x1874671476878464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828833.482659:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476878464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828833.482670:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828833.482674:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828833.482678:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828834.506087:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828835.530065:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828836.553998:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828837.578002:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828838.601959:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828838.601974:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828838.601980:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828838.601994:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3206 00000100:00080000:1.0:1787828838.602009:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828838.602011:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828838.602018:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828838.602070:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476878592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828838.602102:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828838.602105:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828838.602107:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828838.602110:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3206 00000100:00080000:1.0:1787828838.602114:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828838.602120:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828838.602132:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828838.602148:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476878720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828838.602160:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828838.602163:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828838.602165:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828838.602173:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3206 00000100:00100000:2.0:1787828838.602173:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476878592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828838.602179:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828838.602186:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828838.602192:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828838.602206:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476878848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828838.602214:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828838.602217:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828838.602219:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828838.602222:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3206 00000100:00080000:1.0:1787828838.602225:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828838.602228:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828838.602233:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828838.602243:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476878976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828838.602250:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828838.602252:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828838.602254:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828838.602256:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3206 00000100:00080000:1.0:1787828838.602259:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828838.602262:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828838.602267:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787828838.602268:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828838.602277:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476879104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828838.602345:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828838/real 1787828838] req@ffff9a7890b1e580 x1874671476878592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828838.602369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476878592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828838.602382:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828838.602387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828838.602392:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476878592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828838.602437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828838.602483:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828838.602501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828838/real 1787828838] req@ffff9a7890b1d680 x1874671476878720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828838.602516:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476878720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828838.602543:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828838.602547:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828838.602550:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828838.602557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828838.602582:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828838.602596:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828838/real 1787828838] req@ffff9a7890b1de00 x1874671476878848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828838.602610:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476878848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828838.602619:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828838.602623:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828838.602627:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828838.602633:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828838.602657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828838.602671:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828838/real 1787828838] req@ffff9a7890b1e940 x1874671476878976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828838.602684:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476878976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828838.602696:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828838.602699:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828838.602703:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476878976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828838.602710:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828838.602732:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828838.602745:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828838/real 1787828838] req@ffff9a7890b1da40 x1874671476879104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828838.602758:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476879104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828838.602769:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828838.602773:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828838.602776:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828839.625992:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828840.650065:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828841.673973:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828842.697996:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828843.722013:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828843.722027:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828843.722034:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828843.722047:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828843.722049:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3211 00000100:00080000:0.0:1787828843.722078:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828843.722087:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828843.722146:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476879232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828843.722195:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828843.722199:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828843.722202:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828843.722207:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3211 00000100:00080000:0.0:1787828843.722213:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828843.722218:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828843.722232:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828843.722255:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476879360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828843.722257:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476879232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828843.722287:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828843.722290:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828843.722292:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828843.722299:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3211 00000100:00080000:0.0:1787828843.722302:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828843.722305:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828843.722309:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828843.722322:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476879488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828843.722339:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828843.722347:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:2.0:1787828843.722348:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828843.722353:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828843.722359:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3211 00000100:00080000:0.0:1787828843.722382:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828843.722389:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828843.722396:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828843.722412:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476879616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828843.722423:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828843.722424:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828843/real 1787828843] req@ffff9a7890a3fc00 x1874671476879232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828843.722434:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828843.722440:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828843.722457:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3211 00000100:00080000:0.0:1787828843.722465:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828843.722478:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828843.722485:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828843.722491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476879232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828843.722498:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476879744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828843.722517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828843.722528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828843.722536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476879232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828843.722566:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828843.722588:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828843.722599:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828843/real 1787828843] req@ffff9a7890a3f480 x1874671476879360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828843.722608:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476879360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828843.722614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828843.722616:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828843.722619:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828843.722623:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828843.722635:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828843.722641:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828843/real 1787828843] req@ffff9a7890a3c3c0 x1874671476879488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828843.722648:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476879488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828843.722658:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828843.722660:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828843.722663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828843.722666:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828843.722677:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828843.722683:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828843/real 1787828843] req@ffff9a7890a3c000 x1874671476879616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828843.722690:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476879616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828843.722697:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828843.722699:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828843.722701:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828843.722705:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828843.722724:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828843.722730:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828843/real 1787828843] req@ffff9a7890a3da40 x1874671476879744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828843.722737:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476879744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828843.722744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828843.722746:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828843.722749:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476879744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828844.746089:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828845.770031:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828846.794063:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828847.818163:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828848.841969:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828848.841979:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828848.841984:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828848.841987:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828848.841997:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3216 00000100:00080000:3.0:1787828848.842010:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828848.842020:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828848.842066:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476879872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828848.842090:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828848.842094:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828848.842096:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828848.842098:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3216 00000100:00080000:3.0:1787828848.842101:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828848.842103:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828848.842112:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828848.842123:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476880000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828848.842132:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828848.842135:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828848.842136:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828848.842139:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3216 00000100:00080000:3.0:1787828848.842141:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828848.842148:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828848.842150:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476879872:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787828848.842152:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828848.842171:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476880128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828848.842179:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828848.842193:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828848.842194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828848.842198:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3216 00000100:00080000:3.0:1787828848.842200:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828848.842203:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828848.842208:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828848.842217:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476880256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828848.842224:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828848.842227:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828848.842228:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828848.842231:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3216 00000100:00080000:3.0:1787828848.842258:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828848.842264:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828848.842269:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787828848.842274:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828848.842290:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476880384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828848.842382:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828848/real 1787828848] req@ffff9a78a6776d00 x1874671476879872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828848.842410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476879872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828848.842426:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828848.842433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828848.842439:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476879872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828848.842470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828848.842506:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828848.842538:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828848/real 1787828848] req@ffff9a78a6775680 x1874671476880000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828848.842553:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476880000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828848.842563:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828848.842567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828848.842570:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828848.842578:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828848.842602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828848.842616:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828848/real 1787828848] req@ffff9a78a6775e00 x1874671476880128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828848.842630:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476880128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828848.842639:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828848.842643:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828848.842647:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828848.842654:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828848.842676:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828848.842689:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828848/real 1787828848] req@ffff9a78a6777840 x1874671476880256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828848.842702:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476880256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828848.842713:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828848.842717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828848.842721:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828848.842728:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828848.842749:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828848.842760:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828848/real 1787828848] req@ffff9a78a6774f00 x1874671476880384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828848.842780:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476880384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828848.842792:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828848.842796:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828848.842799:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828849.865974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828850.889958:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828851.914006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828852.938100:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828853.961929:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828853.961937:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828853.961941:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828853.961950:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3221 00000100:00080000:0.0:1787828853.961959:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828853.961965:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828853.962003:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828853.962005:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476880512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828853.962059:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828853.962065:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828853.962068:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828853.962074:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3221 00000100:00080000:0.0:1787828853.962079:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828853.962087:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828853.962100:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828853.962139:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476880512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828853.962141:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476880640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828853.962164:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828853.962168:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828853.962171:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828853.962176:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3221 00000100:00080000:0.0:1787828853.962181:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828853.962186:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828853.962192:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828853.962219:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476880768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828853.962239:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828853.962243:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828853.962246:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828853.962253:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3221 00000100:00080000:0.0:1787828853.962263:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828853.962285:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828853.962290:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828853.962313:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787828853.962321:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476880896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828853.962333:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828853.962336:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828853.962339:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828853.962343:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3221 00000100:00080000:0.0:1787828853.962347:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828853.962351:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880768:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787828853.962355:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828853.962363:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828853.962377:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476881024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828853.962390:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828853.962415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828854.986024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828856.010055:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828857.034097:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787828857.034168:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787828857.034205:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787828857.034218:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828857.034249:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828857.034260:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828857.034266:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828857.034285:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787828857.034357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828853/real 1787828857] req@ffff9a7890a3f480 x1874671476881024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828857.034395:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476881024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828857.034410:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828857.034415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828857.034420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881024:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828857.034439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828853/real 1787828857] req@ffff9a7890a3c3c0 x1874671476880896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828857.034452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476880896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828857.034462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828857.034466:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828857.034475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828857.034483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828853/real 1787828857] req@ffff9a7890a3da40 x1874671476880640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828857.034499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476880640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828857.034511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828857.034515:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828857.034519:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828857.034529:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828853/real 1787828857] req@ffff9a7890a3c000 x1874671476880768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828857.034541:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476880768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828857.034550:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828857.034553:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828857.034557:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476880768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828857.034564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828853/real 1787828857] req@ffff9a7890a3de00 x1874671476880512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828857.034577:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476880512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828878 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828857.034585:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828857.034588:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828857.034592:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476880512:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787828858.058058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828859.081973:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828859.081984:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828859.081990:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828859.082001:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3226 00000100:00080000:0.0:1787828859.082018:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787828859.082018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828859.082028:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828859.082081:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476881152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828859.082107:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828859.082111:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828859.082112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828859.082115:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3226 00000100:00080000:0.0:1787828859.082118:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828859.082120:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828859.082130:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828859.082145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476881280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828859.082159:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476881152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828859.082160:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828859.082170:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828859.082172:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828859.082177:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3226 00000100:00080000:0.0:1787828859.082181:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828859.082186:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828859.082192:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828859.082214:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476881408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828859.082224:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828859.082229:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828859.082231:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828859.082236:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3226 00000100:00080000:0.0:1787828859.082240:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828859.082244:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828859.082251:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787828859.082255:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828859.082280:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476881536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828859.082292:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828859.082296:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828859.082299:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828859.082304:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3226 00000100:00080000:0.0:1787828859.082308:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828859.082313:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828859.082319:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828859.082333:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476881664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828859.082345:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff9a7890a3fc00 x1874671476881152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828859.082371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476881152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828859.082382:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828859.082388:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828859.082393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476881152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828859.082426:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828859.082452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828859.082466:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff9a7890a3de00 x1874671476881280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828859.082486:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476881280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828859.082495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828859.082498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828859.082501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828859.082508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828859.082530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828859.082541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff9a7890a3c000 x1874671476881408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828859.082552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476881408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828859.082559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828859.082562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828859.082565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828859.082577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828859.082598:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828859.082615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff9a7890a3da40 x1874671476881536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828859.082627:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476881536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828859.082644:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828859.082647:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828859.082650:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828859.082656:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828859.082677:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828859.082688:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828859/real 1787828859] req@ffff9a7890a3c3c0 x1874671476881664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828859.082700:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476881664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828859.082711:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828859.082716:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828859.082719:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828860.106082:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828861.130046:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828862.153995:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828863.178012:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828864.201964:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828864.201972:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828864.201976:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828864.201985:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3231 00000100:00080000:2.0:1787828864.202007:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828864.202013:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828864.202015:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828864.202052:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476881792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828864.202094:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828864.202097:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828864.202098:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828864.202101:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3231 00000100:00080000:2.0:1787828864.202103:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828864.202106:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828864.202114:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828864.202124:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476881920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828864.202132:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828864.202135:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828864.202136:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828864.202139:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3231 00000100:00080000:2.0:1787828864.202141:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828864.202152:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828864.202154:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476881792:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828864.202156:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828864.202168:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476882048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828864.202181:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828864.202183:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828864.202184:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828864.202189:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3231 00000100:00080000:2.0:1787828864.202195:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828864.202197:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828864.202200:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828864.202211:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476882176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828864.202218:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828864.202220:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828864.202221:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828864.202238:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3231 00000100:00080000:2.0:1787828864.202257:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828864.202261:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828864.202265:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787828864.202274:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828864.202275:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476882304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828864.202356:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff9a78841da940 x1874671476881792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828864.202388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476881792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828864.202402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828864.202408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828864.202414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476881792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828864.202461:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828864.202490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828864.202531:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff9a78841d9680 x1874671476881920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828864.202547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476881920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828864.202556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828864.202561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828864.202565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476881920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828864.202572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828864.202594:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828864.202607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff9a78841d92c0 x1874671476882048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828864.202620:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476882048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828864.202637:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828864.202641:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828864.202645:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828864.202652:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828864.202677:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828864.202706:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff9a78841db840 x1874671476882176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828864.202720:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476882176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828864.202731:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828864.202735:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828864.202739:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828864.202754:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828864.202776:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828864.202790:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828864/real 1787828864] req@ffff9a78841d8b40 x1874671476882304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828864.202803:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476882304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828864.202814:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828864.202818:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828864.202823:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828865.225966:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828866.250006:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828867.274071:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828868.297988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828869.321913:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.321923:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828869.321927:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.321938:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3236 00000100:00080000:0.0:1787828869.321948:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.321955:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828869.321987:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476882432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828869.322013:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.322019:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828869.322021:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.322026:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3236 00000100:00080000:0.0:1787828869.322029:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.322037:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828869.322042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00100000:0.0:1787828869.322047:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828869.322057:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476882432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828869.322076:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476882560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828869.322089:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.322093:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828869.322095:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.322099:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3236 00000100:00080000:0.0:1787828869.322103:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.322107:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828869.322112:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828869.322122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476882688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828869.322133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828869.322135:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.322140:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828869.322142:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.322148:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3236 00000100:00080000:0.0:1787828869.322159:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.322163:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828869.322169:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828869.322180:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476882816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828869.322184:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff9a7890a3f480 x1874671476882432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828869.322212:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828869.322217:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828869.322220:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828869.322225:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3236 00000100:00100000:3.0:1787828869.322229:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476882432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828869.322244:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828869.322251:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828869.322257:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828869.322257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828869.322267:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787828869.322274:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476882944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828869.322275:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476882432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828869.322306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828869.322331:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828869.322343:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff9a7890a3c3c0 x1874671476882560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828869.322357:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476882560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828869.322367:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828869.322371:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828869.322374:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828869.322380:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828869.322396:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828869.322406:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff9a7890a3da40 x1874671476882688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828869.322418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476882688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828869.322427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828869.322430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828869.322433:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828869.322438:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828869.322452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828869.322459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff9a7890a3c000 x1874671476882816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828869.322467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476882816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828869.322473:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828869.322474:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828869.322476:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828869.322480:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828869.322489:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828869.322494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828869/real 1787828869] req@ffff9a7890a3de00 x1874671476882944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828869.322501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476882944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828869.322507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828869.322509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828869.322511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476882944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828870.346083:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828871.370019:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828872.393959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828873.418037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828874.441985:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.441997:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828874.442002:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.442015:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3241 00000100:00080000:0.0:1787828874.442027:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.442035:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828874.442085:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476883072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828874.442100:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828874.442112:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.442117:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828874.442119:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.442123:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3241 00000100:00080000:0.0:1787828874.442126:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.442130:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828874.442142:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828874.442162:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476883072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828874.442172:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476883200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828874.442181:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.442196:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828874.442197:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.442201:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3241 00000100:00080000:0.0:1787828874.442204:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.442207:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828874.442211:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828874.442221:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476883328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828874.442228:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.442231:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828874.442232:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.442236:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3241 00000100:00080000:0.0:1787828874.442242:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.442244:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828874.442249:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828874.442257:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828874.442258:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476883456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828874.442267:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828874.442270:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828874.442271:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828874.442274:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3241 00000100:00080000:0.0:1787828874.442283:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828874.442285:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828874.442288:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828874.442297:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476883584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828874.442312:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828874] req@ffff9a7890a3fc00 x1874671476883072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828874.442329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476883072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828874.442338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828874.442341:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828874.442344:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476883072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828874.442367:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828874.442383:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828874.442391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828874] req@ffff9a7890a3de00 x1874671476883200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828874.442399:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476883200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828874.442405:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828874.442407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828874.442410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828874.442414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828874.442424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828874.442430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828874] req@ffff9a7890a3c000 x1874671476883328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828874.442438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476883328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828874.442443:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828874.442445:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828874.442447:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828874.442450:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828874.442461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828874.442467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828874] req@ffff9a7890a3da40 x1874671476883456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828874.442474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476883456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828874.442481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828874.442482:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828874.442484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828874.442488:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828874.442497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828874.442502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828874/real 1787828874] req@ffff9a7890a3c3c0 x1874671476883584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828874.442510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476883584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828874.442517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828874.442518:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828874.442520:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828875.466069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828876.490085:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828877.514060:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828878.538023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828879.561986:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828879.561998:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828879.562003:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828879.562026:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3247 00000100:00080000:0.0:1787828879.562038:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828879.562046:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828879.562081:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828879.562094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476883712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828879.562136:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828879.562140:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828879.562142:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828879.562145:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3247 00000100:00080000:0.0:1787828879.562148:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828879.562152:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828879.562163:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828879.562174:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476883840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828879.562184:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828879.562185:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476883712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828879.562188:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828879.562191:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828879.562198:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3247 00000100:00080000:0.0:1787828879.562201:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828879.562204:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828879.562207:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828879.562216:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476883968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828879.562222:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828879.562224:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828879.562226:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828879.562229:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3247 00000100:00080000:0.0:1787828879.562232:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828879.562235:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828879.562239:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828879.562248:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476884096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828879.562255:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828879.562257:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828879.562259:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828879.562262:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3247 00000100:00080000:0.0:1787828879.562265:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828879.562267:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828879.562269:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828879.562279:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476884224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828879.562281:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828879.562347:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828879/real 1787828879] req@ffff9a7890a3f480 x1874671476883712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828879.562368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476883712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828879.562380:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828879.562385:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828879.562390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476883712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828879.562419:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828879.562442:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828879.562455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828879/real 1787828879] req@ffff9a7890a3c3c0 x1874671476883840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828879.562467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476883840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828879.562482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828879.562486:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828879.562489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828879.562496:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828879.562516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828879.562528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828879/real 1787828879] req@ffff9a7890a3da40 x1874671476883968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828879.562542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476883968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828879.562551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828879.562555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828879.562559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476883968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828879.562566:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828879.562585:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828879.562594:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828879/real 1787828879] req@ffff9a7890a3c000 x1874671476884096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828879.562602:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476884096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828879.562610:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828879.562623:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828879.562625:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828879.562629:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828879.562640:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828879.562650:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828879/real 1787828879] req@ffff9a7890a3de00 x1874671476884224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828879.562662:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476884224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828879.562672:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828879.562676:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828879.562690:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828880.585983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828881.610009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828882.634014:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828883.657923:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828884.681976:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828884.682042:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828884.682052:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828884.682058:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828884.682068:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3252 00000100:00080000:2.0:1787828884.682077:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828884.682101:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828884.682153:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476884352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828884.682182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828884.682185:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828884.682186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828884.682189:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3252 00000100:00080000:2.0:1787828884.682191:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828884.682194:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828884.682202:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828884.682211:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476884480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828884.682218:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828884.682221:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828884.682231:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828884.682234:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3252 00000100:00080000:2.0:1787828884.682236:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828884.682241:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828884.682242:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476884352:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787828884.682245:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828884.682255:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476884608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828884.682266:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828884.682268:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828884.682269:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828884.682272:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3252 00000100:00080000:2.0:1787828884.682274:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828884.682277:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828884.682280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828884.682298:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476884736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828884.682304:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828884.682307:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828884.682308:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828884.682313:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3252 00000800:00000100:3.0:1787828884.682314:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828884.682316:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828884.682320:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828884.682323:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828884.682332:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476884864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828884.682383:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828884/real 1787828884] req@ffff9a78841db480 x1874671476884352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828884.682404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476884352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828884.682417:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828884.682421:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828884.682426:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476884352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828884.682433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828884.682453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828884.682463:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828884/real 1787828884] req@ffff9a78841d83c0 x1874671476884480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828884.682472:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476884480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828884.682478:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828884.682480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828884.682482:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828884.682518:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828884.682533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828884.682541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828884/real 1787828884] req@ffff9a78841d8000 x1874671476884608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828884.682550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476884608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828884.682565:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828884.682567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828884.682569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828884.682573:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828884.682602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828884.682609:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828884/real 1787828884] req@ffff9a78841da580 x1874671476884736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828884.682616:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476884736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828884.682634:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828884.682636:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828884.682639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828884.682642:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828884.682652:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828884.682658:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828884/real 1787828884] req@ffff9a78841d9e00 x1874671476884864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828884.682666:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476884864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828884.682673:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828884.682675:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828884.682677:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476884864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828885.705934:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828886.729938:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828887.753966:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828888.777959:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828889.801939:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828889.801951:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828889.801956:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828889.801967:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3257 00000100:00080000:2.0:1787828889.801985:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828889.801994:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828889.802022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828889.802041:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476884992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828889.802089:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828889.802105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828889.802107:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828889.802112:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3257 00000100:00080000:2.0:1787828889.802118:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828889.802124:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828889.802135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828889.802147:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476884992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828889.802154:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476885120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828889.802169:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828889.802172:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828889.802174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828889.802178:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3257 00000100:00080000:2.0:1787828889.802181:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828889.802185:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828889.802191:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828889.802204:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476885248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828889.802232:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828889.802236:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828889.802238:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828889.802247:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828889.802249:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3257 00000100:00080000:2.0:1787828889.802258:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828889.802262:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828889.802268:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828889.802281:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476885376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828889.802292:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828889.802295:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828889.802298:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828889.802302:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3257 00000100:00080000:2.0:1787828889.802306:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828889.802310:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828889.802316:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828889.802331:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828889/real 1787828889] req@ffff9a78841d8780 x1874671476884992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828889.802332:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476885504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828889.802391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476884992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828889.802406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828889.802411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828889.802415:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476884992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828889.802443:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828889.802472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828889.802498:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828889/real 1787828889] req@ffff9a78841d9a40 x1874671476885120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828889.802511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476885120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828889.802520:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828889.802523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828889.802527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828889.802533:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828889.802548:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828889.802574:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828889/real 1787828889] req@ffff9a78841da1c0 x1874671476885248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828889.802586:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476885248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828889.802594:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828889.802598:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828889.802601:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828889.802606:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828889.802627:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828889.802637:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828889/real 1787828889] req@ffff9a78841db0c0 x1874671476885376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828889.802648:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476885376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828889.802657:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828889.802660:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828889.802663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828889.802668:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828889.802693:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828889.802703:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828889/real 1787828889] req@ffff9a78841dbc00 x1874671476885504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828889.802715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476885504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828889.802724:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828889.802727:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828889.802730:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828890.826021:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828891.850119:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828892.873971:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828893.897985:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828894.921938:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787828894.921944:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828894.921946:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828894.921951:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828894.921959:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3262 00000100:00080000:0.0:1787828894.921967:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828894.921973:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828894.922004:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476885632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828894.922039:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828894.922044:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828894.922045:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828894.922048:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3262 00000100:00080000:0.0:1787828894.922063:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828894.922066:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828894.922074:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828894.922084:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476885632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828894.922087:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476885760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828894.922099:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828894.922101:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828894.922102:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828894.922105:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3262 00000100:00080000:0.0:1787828894.922107:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828894.922109:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828894.922112:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828894.922120:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476885888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828894.922126:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828894.922129:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828894.922130:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828894.922133:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3262 00000100:00080000:0.0:1787828894.922141:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828894.922143:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828894.922147:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828894.922154:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828894.922156:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476886016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828894.922164:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828894.922166:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828894.922168:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828894.922171:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3262 00000100:00080000:0.0:1787828894.922173:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828894.922176:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828894.922178:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828894.922184:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476886144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828894.922210:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828894/real 1787828894] req@ffff9a7890a3fc00 x1874671476885632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828894.922224:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476885632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828894.922232:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828894.922235:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828894.922238:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476885632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828894.922267:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828894.922294:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828894.922303:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828894/real 1787828894] req@ffff9a7890a3de00 x1874671476885760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828894.922311:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476885760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828894.922317:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828894.922329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828894.922332:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828894.922335:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828894.922346:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828894.922355:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828894/real 1787828894] req@ffff9a7890a3c000 x1874671476885888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828894.922366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476885888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828894.922375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828894.922378:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828894.922382:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476885888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828894.922388:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828894.922409:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828894.922418:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828894/real 1787828894] req@ffff9a7890a3da40 x1874671476886016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828894.922426:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476886016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828894.922432:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828894.922434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828894.922437:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828894.922440:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828894.922451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828894.922457:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828894/real 1787828894] req@ffff9a7890a3c3c0 x1874671476886144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828894.922464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476886144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828919 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828894.922470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828894.922472:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828894.922474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828895.945962:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828896.970059:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828897.994005:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828899.017942:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828900.041928:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.041936:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828900.041940:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828900.041948:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3267 00000100:00080000:0.0:1787828900.041957:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.041962:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828900.041971:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828900.042011:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476886272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828900.042031:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.042033:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828900.042034:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828900.042037:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3267 00000100:00080000:0.0:1787828900.042040:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.042043:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828900.042050:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828900.042060:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476886400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828900.042068:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.042071:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828900.042074:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828900.042075:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476886272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828900.042078:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3267 00000100:00080000:0.0:1787828900.042082:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.042086:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828900.042089:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828900.042097:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476886528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828900.042103:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.042106:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828900.042107:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828900.042110:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3267 00000100:00080000:0.0:1787828900.042111:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.042113:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828900.042117:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828900.042124:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476886656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828900.042134:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828900.042137:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828900.042138:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828900.042144:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828900.042145:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3267 00000100:00080000:0.0:1787828900.042150:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828900.042153:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828900.042156:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828900.042164:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476886784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828900.042203:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff9a7890a3f480 x1874671476886272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.042223:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476886272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.042234:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.042239:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828900.042243:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476886272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828900.042266:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828900.042289:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828900.042302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff9a7890a3c3c0 x1874671476886400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.042316:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476886400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.042326:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.042329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828900.042333:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828900.042340:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828900.042360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828900.042371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff9a7890a3da40 x1874671476886528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.042384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476886528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.042399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.042403:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828900.042407:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828900.042413:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828900.042433:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828900.042444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff9a7890a3c000 x1874671476886656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.042456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476886656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.042466:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.042469:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828900.042473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828900.042479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828900.042498:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828900.042509:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828900/real 1787828900] req@ffff9a7890a3de00 x1874671476886784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828900.042522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476886784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828900.042532:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828900.042535:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828900.042539:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476886784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828901.065961:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828902.090016:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828903.114031:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828904.138053:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828905.162052:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828905.162068:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828905.162068:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828905.162084:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828905.162104:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3272 00000100:00080000:2.0:1787828905.162122:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828905.162135:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828905.162201:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476886912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828905.162242:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828905.162246:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828905.162248:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828905.162253:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3272 00000100:00080000:2.0:1787828905.162257:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828905.162263:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828905.162276:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828905.162292:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476887040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828905.162305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476886912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828905.162311:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828905.162324:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828905.162326:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828905.162332:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3272 00000100:00080000:2.0:1787828905.162337:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828905.162343:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828905.162349:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828905.162378:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476887168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828905.162389:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828905.162393:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828905.162394:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828905.162398:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3272 00000100:00080000:2.0:1787828905.162425:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828905.162429:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828905.162434:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828905.162445:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828905.162453:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476887296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828905.162473:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828905.162480:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828905.162484:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828905.162492:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3272 00000100:00080000:2.0:1787828905.162500:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828905.162508:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828905.162520:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828905.162550:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476887424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828905.162574:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff9a78841dbc00 x1874671476886912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.162611:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476886912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.162626:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.162632:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828905.162637:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476886912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828905.162696:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828905.162729:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828905.162745:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff9a78841db0c0 x1874671476887040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.162759:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476887040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.162769:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.162772:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828905.162793:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828905.162801:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828905.162820:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828905.162831:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff9a78841da1c0 x1874671476887168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.162868:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476887168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.162878:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.162881:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828905.162885:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828905.162891:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828905.162911:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828905.162922:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff9a78841d9a40 x1874671476887296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.162934:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476887296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.162971:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.162975:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828905.162979:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828905.162986:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828905.163009:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828905.163021:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828905/real 1787828905] req@ffff9a78841d8780 x1874671476887424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828905.163033:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476887424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828905.163042:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828905.163045:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828905.163049:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828906.186056:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828907.210045:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828908.233992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828909.258134:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828910.282032:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.282046:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828910.282052:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.282065:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3277 00010000:00010000:2.0:1787828910.282073:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828910.282078:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.282087:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828910.282147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476887552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828910.282180:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.282184:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828910.282186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.282190:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3277 00000100:00080000:0.0:1787828910.282194:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.282198:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828910.282208:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828910.282209:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476887552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828910.282225:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476887680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828910.282236:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.282239:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828910.282241:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.282245:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3277 00000100:00080000:0.0:1787828910.282261:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.282264:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828910.282268:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828910.282272:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828910.282282:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476887808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828910.282292:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.282295:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828910.282296:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.282299:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3277 00000100:00080000:0.0:1787828910.282302:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.282306:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828910.282310:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828910.282321:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476887936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828910.282329:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828910.282334:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787828910.282335:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff9a7890a3fc00 x1874671476887552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828910.282337:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828910.282346:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3277 00000100:00080000:0.0:1787828910.282353:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828910.282357:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828910.282362:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828910.282367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476887552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828910.282376:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476888064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828910.282383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828910.282393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828910.282397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476887552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828910.282425:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828910.282445:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828910.282456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff9a7890a3de00 x1874671476887680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828910.282466:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476887680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828910.282473:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828910.282476:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828910.282479:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828910.282484:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828910.282497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828910.282505:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff9a7890a3c000 x1874671476887808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828910.282514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476887808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828910.282520:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828910.282522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828910.282525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828910.282529:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828910.282541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828910.282548:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff9a7890a3da40 x1874671476887936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828910.282562:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476887936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828910.282573:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828910.282589:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828910.282594:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476887936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828910.282603:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828910.282630:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828910.282647:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828910/real 1787828910] req@ffff9a7890a3c3c0 x1874671476888064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828910.282664:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476888064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828910.282678:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828910.282682:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828910.282687:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828911.306037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828912.330024:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828913.354072:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828914.378030:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828915.401966:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.401976:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828915.401981:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828915.401992:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3282 00000100:00080000:0.0:1787828915.402004:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.402011:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828915.402016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828915.402056:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476888192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828915.402080:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.402083:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828915.402084:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828915.402087:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3282 00000100:00080000:0.0:1787828915.402089:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.402092:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828915.402101:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828915.402112:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476888320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828915.402119:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476888192:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828915.402134:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.402137:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828915.402137:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828915.402141:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3282 00000100:00080000:0.0:1787828915.402143:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.402145:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828915.402148:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828915.402157:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476888448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828915.402163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.402165:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828915.402167:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828915.402169:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3282 00000100:00080000:0.0:1787828915.402184:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.402187:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828915.402190:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828915.402195:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828915.402200:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476888576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828915.402207:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828915.402210:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828915.402211:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828915.402214:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3282 00000100:00080000:0.0:1787828915.402220:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828915.402222:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828915.402225:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828915.402232:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476888704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828915.402250:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff9a7890a3f480 x1874671476888192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828915.402271:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476888192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828915.402283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828915.402288:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828915.402294:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476888192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828915.402323:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828915.402349:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828915.402364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff9a7890a3c3c0 x1874671476888320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828915.402374:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476888320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828915.402385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828915.402388:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828915.402390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828915.402395:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828915.402410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828915.402417:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff9a7890a3da40 x1874671476888448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828915.402425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476888448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828915.402430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828915.402432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828915.402434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828915.402438:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828915.402449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828915.402456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff9a7890a3c000 x1874671476888576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828915.402464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476888576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828915.402470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828915.402472:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828915.402475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828915.402478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828915.402488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828915.402493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828915/real 1787828915] req@ffff9a7890a3de00 x1874671476888704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828915.402500:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476888704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828915.402507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828915.402509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828915.402511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828916.426041:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828917.450001:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828918.473997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828919.498058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828920.522025:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828920.522036:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828920.522040:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828920.522053:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3288 00000100:00080000:0.0:1787828920.522068:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828920.522078:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828920.522104:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828920.522135:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476888832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828920.522162:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828920.522165:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828920.522166:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828920.522170:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3288 00000100:00080000:0.0:1787828920.522173:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828920.522175:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828920.522184:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828920.522197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476888960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828920.522205:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828920.522210:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828920.522212:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828920.522215:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3288 00000100:00080000:0.0:1787828920.522217:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828920.522220:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828920.522226:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828920.522237:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476889088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828920.522239:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476888832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828920.522251:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828920.522257:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828920.522260:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828920.522263:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3288 00000100:00080000:0.0:1787828920.522265:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828920.522267:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828920.522270:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828920.522279:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476889216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828920.522285:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828920.522287:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828920.522288:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828920.522291:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3288 00000100:00080000:0.0:1787828920.522294:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828920.522297:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828920.522301:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828920.522308:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476889344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828920.522430:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828920.522508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828920.522551:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828920.522589:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828921.545976:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828922.570024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828923.594023:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828923.594154:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787828923.594197:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828923.594209:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828923.594246:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828923.594257:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828923.594267:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787828923.594277:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787828923.594291:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828920/real 1787828923] req@ffff9a7890a3c000 x1874671476889088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828923.594324:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476889088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828923.594342:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828923.594347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828923.594352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828923.594359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828920/real 1787828923] req@ffff9a7890a3da40 x1874671476889216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828923.594373:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476889216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828923.594382:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828923.594386:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828923.594390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828923.594396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828920/real 1787828923] req@ffff9a7890a3c3c0 x1874671476889344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828923.594406:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476889344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828923.594416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828923.594419:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828923.594423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889344:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828923.594430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828920/real 1787828923] req@ffff9a7890a3fc00 x1874671476888832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828923.594450:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476888832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828923.594459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828923.594462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828923.594466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476888832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828923.594473:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828920/real 1787828923] req@ffff9a7890a3de00 x1874671476888960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828923.594484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476888960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828923.594492:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828923.594495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828923.594499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476888960:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787828924.617982:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828925.641948:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828925.641956:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828925.641959:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828925.641964:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828925.641968:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3293 00000100:00080000:1.0:1787828925.641978:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828925.641989:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828925.642020:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476889472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828925.642038:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828925.642041:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828925.642042:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828925.642045:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3293 00000100:00080000:1.0:1787828925.642047:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828925.642050:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828925.642057:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828925.642077:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476889600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828925.642087:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476889472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828925.642091:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828925.642096:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828925.642098:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828925.642104:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3293 00000100:00080000:1.0:1787828925.642109:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828925.642113:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828925.642119:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828925.642133:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476889728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828925.642142:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828925.642146:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828925.642148:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828925.642154:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787828925.642155:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3293 00000100:00080000:1.0:1787828925.642163:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828925.642168:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828925.642174:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828925.642187:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476889856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828925.642196:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828925.642200:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828925.642202:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828925.642208:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3293 00000100:00100000:3.0:1787828925.642212:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828925/real 1787828925] req@ffff9a7890b1da40 x1874671476889472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828925.642214:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828925.642223:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828925.642234:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828925.642242:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476889472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828925.642249:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476889984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828925.642264:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828925.642271:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828925.642276:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476889472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828925.642302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828925.642332:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828925.642353:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828925/real 1787828925] req@ffff9a7890b1e940 x1874671476889600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828925.642367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476889600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828925.642376:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828925.642380:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828925.642383:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828925.642389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828925.642417:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828925.642430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828925/real 1787828925] req@ffff9a7890b1de00 x1874671476889728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828925.642441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476889728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828925.642448:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828925.642451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828925.642455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828925.642461:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828925.642482:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828925.642493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828925/real 1787828925] req@ffff9a7890b1d680 x1874671476889856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828925.642510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476889856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828925.642519:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828925.642522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828925.642526:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828925.642531:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828925.642550:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828925.642562:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828925/real 1787828925] req@ffff9a7890b1e580 x1874671476889984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828925.642573:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476889984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828925.642582:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828925.642586:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828925.642589:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476889984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828926.665986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828927.690126:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828928.713951:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828929.738036:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828930.761987:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828930.761999:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828930.762004:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828930.762030:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3298 00000100:00080000:0.0:1787828930.762045:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828930.762048:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828930.762054:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828930.762124:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476890112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828930.762147:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828930.762151:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828930.762152:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828930.762156:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3298 00000100:00080000:0.0:1787828930.762159:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828930.762162:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828930.762174:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828930.762187:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476890112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828930.762189:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476890240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828930.762201:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828930.762203:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828930.762204:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828930.762208:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3298 00000100:00080000:0.0:1787828930.762209:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828930.762212:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828930.762216:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828930.762224:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476890368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828930.762232:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828930.762235:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828930.762237:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828930.762240:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3298 00000100:00080000:0.0:1787828930.762253:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828930.762255:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828930.762259:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828930.762266:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828930.762270:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476890496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828930.762278:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828930.762281:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828930.762283:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828930.762286:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3298 00000100:00080000:0.0:1787828930.762290:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828930.762292:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828930.762295:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828930.762303:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476890624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828930.762324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828930] req@ffff9a7890a3f480 x1874671476890112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828930.762340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476890112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828930.762354:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828930.762357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828930.762361:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476890112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828930.762389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828930.762414:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828930.762428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828930] req@ffff9a7890a3de00 x1874671476890240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828930.762439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476890240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828930.762446:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828930.762448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828930.762450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828930.762455:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828930.762470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828930.762477:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828930] req@ffff9a7890a3fc00 x1874671476890368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828930.762484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476890368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828930.762490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828930.762492:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828930.762495:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828930.762499:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828930.762509:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828930.762515:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828930] req@ffff9a7890a3c3c0 x1874671476890496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828930.762522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476890496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828930.762529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828930.762530:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828930.762532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828930.762536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828930.762545:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828930.762551:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828930] req@ffff9a7890a3da40 x1874671476890624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828930.762558:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476890624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828930.762594:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828930.762596:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828930.762598:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828931.786014:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828932.810001:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828933.833977:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828934.857989:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828935.881968:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.881976:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828935.881980:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.881988:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3303 00000100:00080000:0.0:1787828935.881997:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828935.882021:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828935.882037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828935.882085:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476890752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828935.882114:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.882118:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828935.882120:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.882124:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3303 00000100:00080000:0.0:1787828935.882128:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828935.882132:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828935.882141:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828935.882146:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476890752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787828935.882157:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476890880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828935.882172:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.882175:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828935.882177:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.882180:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3303 00000100:00080000:0.0:1787828935.882196:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828935.882198:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828935.882202:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828935.882205:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828935.882219:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476891008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828935.882227:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.882230:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828935.882231:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.882234:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3303 00000100:00080000:0.0:1787828935.882237:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828935.882240:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828935.882244:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828935.882251:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9a7890a3c000 x1874671476890752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828935.882255:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476891136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828935.882282:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476890752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828935.882284:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.882293:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828935.882296:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.882305:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3303 00000100:00080000:3.0:1787828935.882305:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828935.882309:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828935.882312:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787828935.882317:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828935.882322:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476890752:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787828935.882323:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787828935.882337:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476891264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828935.882362:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828935.882382:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828935.882392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9a7890a3da40 x1874671476890880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828935.882401:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476890880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828935.882407:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828935.882409:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828935.882411:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476890880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828935.882415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828935.882426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828935.882432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9a7890a3c3c0 x1874671476891008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828935.882440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476891008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828935.882450:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828935.882452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828935.882455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828935.882458:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828935.882468:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828935.882474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9a7890a3fc00 x1874671476891136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828935.882482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476891136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828935.882488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828935.882490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828935.882493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828935.882518:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828935.882533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828935.882544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9a7890a3de00 x1874671476891264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828935.882559:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476891264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828935.882590:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828935.882592:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828935.882595:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828936.906006:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828937.930118:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828938.953986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828939.978038:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828941.001902:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828941.001910:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828941.001914:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828941.001923:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3308 00000100:00080000:0.0:1787828941.001932:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828941.001939:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787828941.001956:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787828941.001973:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476891392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828941.001994:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828941.001997:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828941.001998:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828941.002001:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3308 00000100:00080000:0.0:1787828941.002003:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828941.002006:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828941.002014:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828941.002026:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476891520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828941.002026:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476891392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787828941.002042:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828941.002048:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828941.002051:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828941.002055:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3308 00000100:00080000:0.0:1787828941.002059:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828941.002063:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828941.002069:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828941.002084:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476891648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828941.002084:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787828941.002097:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828941.002100:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828941.002102:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828941.002106:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3308 00000100:00080000:0.0:1787828941.002112:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828941.002114:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828941.002118:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828941.002129:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476891776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828941.002139:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828941.002144:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828941] req@ffff9a7890a3f480 x1874671476891392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828941.002145:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828941.002149:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828941.002158:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3308 00000100:00080000:0.0:1787828941.002166:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828941.002171:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828941.002175:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476891392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787828941.002178:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787828941.002191:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787828941.002196:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476891904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828941.002197:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828941.002204:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476891392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828941.002231:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828941.002250:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828941.002259:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828941] req@ffff9a7890a3de00 x1874671476891520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828941.002267:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476891520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828941.002272:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828941.002275:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828941.002277:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828941.002280:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828941.002289:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828941.002296:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828941] req@ffff9a7890a3fc00 x1874671476891648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828941.002302:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476891648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828941.002307:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828941.002309:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828941.002311:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828941.002314:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828941.002324:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828941.002329:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828941] req@ffff9a7890a3c3c0 x1874671476891776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828941.002336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476891776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828941.002342:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828941.002343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828941.002345:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828941.002348:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828941.002357:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828941.002362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828941/real 1787828941] req@ffff9a7890a3da40 x1874671476891904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828941.002369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476891904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828941.002375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828941.002377:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828941.002379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476891904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828942.026075:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828943.050127:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828944.074063:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828945.098032:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828946.121933:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828946.121942:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828946.121946:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828946.121959:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3313 00000100:00080000:2.0:1787828946.121970:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828946.121979:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828946.122006:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828946.122018:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476892032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828946.122042:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828946.122045:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828946.122047:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828946.122050:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3313 00000100:00080000:2.0:1787828946.122054:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828946.122057:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828946.122067:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828946.122077:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476892160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828946.122087:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476892032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828946.122091:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828946.122102:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828946.122104:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828946.122109:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3313 00000100:00080000:2.0:1787828946.122111:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828946.122114:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828946.122118:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828946.122128:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476892288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828946.122137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828946.122143:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828946.122146:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828946.122151:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3313 00000100:00080000:2.0:1787828946.122168:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828946.122175:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828946.122181:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828946.122190:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828946.122215:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476892416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828946.122230:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828946.122234:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828946.122235:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828946.122239:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3313 00000100:00080000:2.0:1787828946.122244:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828946.122249:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828946.122254:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828946.122261:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff9a78841d9e00 x1874671476892032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828946.122270:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476892544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828946.122324:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476892032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828946.122341:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828946.122346:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828946.122350:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476892032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828946.122380:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828946.122406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828946.122421:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff9a78841da580 x1874671476892160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828946.122431:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476892160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828946.122437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828946.122440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828946.122443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828946.122448:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828946.122463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828946.122470:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff9a78841d8000 x1874671476892288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828946.122478:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476892288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828946.122483:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828946.122485:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828946.122487:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828946.122491:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828946.122501:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828946.122507:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff9a78841d83c0 x1874671476892416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828946.122514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476892416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828946.122521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828946.122523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828946.122525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828946.122529:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828946.122538:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828946.122543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828946/real 1787828946] req@ffff9a78841db480 x1874671476892544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828946.122551:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476892544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828946.122558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828946.122560:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828946.122562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828947.145983:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828948.169959:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828949.194031:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828950.217949:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828951.241968:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787828951.241971:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828951.241979:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828951.241983:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828951.241993:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3318 00000100:00080000:0.0:1787828951.242003:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828951.242010:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787828951.242070:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476892672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828951.242094:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828951.242097:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828951.242098:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828951.242101:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3318 00000100:00080000:0.0:1787828951.242104:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828951.242107:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828951.242115:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828951.242128:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476892800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828951.242138:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828951.242141:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828951.242143:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828951.242146:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3318 00000100:00080000:0.0:1787828951.242152:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828951.242159:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828951.242159:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476892672:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787828951.242172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828951.242182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476892928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828951.242192:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828951.242195:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787828951.242197:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828951.242201:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3318 00000100:00080000:0.0:1787828951.242203:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828951.242205:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828951.242209:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787828951.242218:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476893056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828951.242224:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828951.242227:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787828951.242228:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828951.242232:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3318 00000100:00080000:0.0:1787828951.242235:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828951.242238:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787828951.242241:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787828951.242245:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787828951.242251:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476893184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828951.242313:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff9a7890a3c000 x1874671476892672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828951.242336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476892672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828951.242347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828951.242350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828951.242354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476892672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828951.242397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828951.242421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828951.242431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff9a7890a3da40 x1874671476892800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828951.242441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476892800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828951.242465:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828951.242468:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828951.242472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828951.242479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828951.242502:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828951.242514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff9a7890a3c3c0 x1874671476892928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828951.242524:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476892928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828951.242530:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828951.242533:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828951.242535:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476892928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828951.242540:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828951.242556:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828951.242563:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff9a7890a3fc00 x1874671476893056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828951.242571:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476893056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828951.242588:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828951.242590:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828951.242593:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828951.242597:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828951.242607:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828951.242613:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828951/real 1787828951] req@ffff9a7890a3de00 x1874671476893184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828951.242638:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476893184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828951.242645:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828951.242647:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828951.242650:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828952.265975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828953.289952:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828954.313945:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828955.337969:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828956.361964:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828956.361981:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828956.361986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828956.361998:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3323 00000100:00080000:2.0:1787828956.362010:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828956.362013:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828956.362018:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828956.362062:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476893312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828956.362086:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828956.362090:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828956.362091:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828956.362094:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3323 00000100:00080000:2.0:1787828956.362097:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828956.362101:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828956.362112:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828956.362119:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476893312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828956.362126:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476893440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828956.362135:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828956.362138:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828956.362139:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828956.362143:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3323 00000100:00080000:2.0:1787828956.362145:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828956.362148:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828956.362150:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828956.362161:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476893568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828956.362168:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828956.362171:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828956.362173:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828956.362175:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3323 00000100:00080000:2.0:1787828956.362184:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828956.362187:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828956.362191:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828956.362201:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476893696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828956.362203:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828956.362211:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828956.362214:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828956.362216:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828956.362220:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3323 00000100:00080000:2.0:1787828956.362222:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828956.362225:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828956.362228:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828956.362236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476893824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828956.362264:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff9a78841d8b40 x1874671476893312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828956.362279:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476893312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828956.362289:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828956.362292:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828956.362295:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476893312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828956.362337:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828956.362353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828956.362362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff9a78841db840 x1874671476893440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828956.362370:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476893440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828956.362376:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828956.362379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828956.362381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828956.362385:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828956.362395:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828956.362401:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff9a78841d92c0 x1874671476893568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828956.362415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476893568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828956.362420:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828956.362423:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828956.362425:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828956.362428:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828956.362449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828956.362455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff9a78841d9680 x1874671476893696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828956.362462:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476893696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828956.362470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828956.362472:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828956.362474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828956.362478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828956.362486:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828956.362492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828956/real 1787828956] req@ffff9a78841da940 x1874671476893824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828956.362499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476893824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828956.362505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828956.362507:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828956.362509:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476893824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828957.386039:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828958.409996:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828959.433951:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828960.457975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828961.481946:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828961.481958:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828961.481963:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828961.481974:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3328 00000100:00080000:2.0:1787828961.481986:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828961.481994:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828961.482029:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828961.482039:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476893952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828961.482064:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828961.482068:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828961.482070:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828961.482073:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3328 00000100:00080000:2.0:1787828961.482076:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828961.482079:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828961.482089:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828961.482098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476894080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828961.482106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828961.482114:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476893952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828961.482117:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828961.482123:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828961.482127:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3328 00000100:00080000:2.0:1787828961.482129:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828961.482132:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828961.482136:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828961.482146:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476894208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828961.482153:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828961.482156:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828961.482158:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828961.482160:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3328 00000100:00080000:2.0:1787828961.482162:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828961.482165:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828961.482168:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828961.482176:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476894336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828961.482183:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828961.482185:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828961.482186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828961.482189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3328 00000100:00080000:2.0:1787828961.482191:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828961.482196:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787828961.482197:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787828961.482200:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828961.482209:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476894464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828961.482248:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828961/real 1787828961] req@ffff9a78841da940 x1874671476893952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828961.482263:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476893952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828961.482272:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828961.482275:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828961.482279:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476893952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828961.482301:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828961.482316:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828961.482325:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828961/real 1787828961] req@ffff9a78841d9680 x1874671476894080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828961.482333:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476894080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828961.482338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828961.482340:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828961.482342:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828961.482346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828961.482356:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828961.482362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828961/real 1787828961] req@ffff9a78841d92c0 x1874671476894208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828961.482369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476894208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828961.482375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828961.482376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828961.482378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828961.482382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828961.482392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828961.482399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828961/real 1787828961] req@ffff9a78841db840 x1874671476894336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828961.482406:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476894336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828961.482413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828961.482415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828961.482417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828961.482420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828961.482430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828961.482436:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828961/real 1787828961] req@ffff9a78841d8b40 x1874671476894464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828961.482443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476894464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828961.482449:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828961.482451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828961.482453:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828962.505999:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828963.530004:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828964.553951:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828965.578068:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828966.601906:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828966.601919:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828966.601925:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828966.601954:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3334 00000100:00080000:2.0:1787828966.601966:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828966.601981:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787828966.602031:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787828966.602039:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476894592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828966.602064:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828966.602067:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828966.602069:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828966.602073:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3334 00000100:00080000:2.0:1787828966.602077:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828966.602082:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828966.602092:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828966.602103:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476894720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828966.602109:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476894592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787828966.602120:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828966.602123:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828966.602125:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828966.602129:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3334 00000100:00080000:2.0:1787828966.602131:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828966.602134:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828966.602138:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828966.602150:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476894848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828966.602158:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828966.602162:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828966.602163:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828966.602166:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3334 00000100:00080000:2.0:1787828966.602172:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828966.602174:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828966.602180:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787828966.602184:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828966.602195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476894976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828966.602210:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828966.602213:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828966.602215:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828966.602219:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3334 00000100:00080000:2.0:1787828966.602221:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828966.602224:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828966.602228:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828966.602240:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476895104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828966.602248:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828966/real 1787828966] req@ffff9a78841db480 x1874671476894592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828966.602285:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476894592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828966.602295:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828966.602299:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828966.602303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476894592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828966.602325:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828966.602342:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828966.602351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828966/real 1787828966] req@ffff9a78841d83c0 x1874671476894720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828966.602360:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476894720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828966.602366:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828966.602368:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828966.602371:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828966.602375:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828966.602386:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828966.602408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828966/real 1787828966] req@ffff9a78841d8000 x1874671476894848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828966.602418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476894848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828966.602425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828966.602428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828966.602431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828966.602436:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828966.602450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828966.602458:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828966/real 1787828966] req@ffff9a78841da580 x1874671476894976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828966.602467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476894976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828966.602475:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828966.602477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828966.602480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476894976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828966.602484:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828966.602496:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828966.602502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828966/real 1787828966] req@ffff9a78841d9e00 x1874671476895104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828966.602533:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476895104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828966.602542:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828966.602545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828966.602548:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828967.626010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828968.650026:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828969.674021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828970.698045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828971.721975:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828971.721988:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828971.721994:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828971.722007:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3339 00000100:00080000:3.0:1787828971.722020:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828971.722029:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828971.722078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476895232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828971.722112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828971.722116:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828971.722118:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828971.722122:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3339 00000100:00080000:3.0:1787828971.722126:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828971.722131:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828971.722143:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828971.722159:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476895360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828971.722160:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476895232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828971.722179:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828971.722183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828971.722184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828971.722207:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3339 00000100:00080000:3.0:1787828971.722211:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828971.722213:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828971.722220:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828971.722234:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476895488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828971.722250:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828971.722253:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828971.722255:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828971.722259:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3339 00000100:00080000:3.0:1787828971.722279:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828971.722285:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787828971.722287:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787828971.722292:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828971.722306:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476895616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828971.722316:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828971.722320:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828971.722322:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828971.722325:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3339 00000100:00080000:3.0:1787828971.722338:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828971.722342:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828971.722346:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828971.722358:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476895744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828971.722370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828971/real 1787828971] req@ffff9a78a6777c00 x1874671476895232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828971.722392:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476895232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828971.722405:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828971.722409:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828971.722415:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476895232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828971.722446:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828971.722467:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828971.722498:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828971/real 1787828971] req@ffff9a78a67743c0 x1874671476895360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828971.722510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476895360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828971.722529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828971.722532:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828971.722535:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828971.722540:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828971.722555:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828971.722564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828971/real 1787828971] req@ffff9a78a6774f00 x1874671476895488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828971.722574:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476895488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828971.722580:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828971.722583:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828971.722585:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828971.722590:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828971.722604:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828971.722611:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828971/real 1787828971] req@ffff9a78a6777840 x1874671476895616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828971.722620:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476895616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828971.722635:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828971.722638:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828971.722641:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828971.722645:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828971.722657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828971.722664:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828971/real 1787828971] req@ffff9a78a6775e00 x1874671476895744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828971.722673:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476895744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787828996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828971.722681:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828971.722684:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828971.722686:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476895744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828971.722720:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828972.746145:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828973.770171:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828974.794213:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828975.817975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828976.842004:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828976.842016:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:1.0:1787828976.842021:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828976.842022:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828976.842046:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3344 00000100:00080000:2.0:1787828976.842059:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828976.842066:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828976.842131:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476895872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828976.842159:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828976.842163:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828976.842164:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828976.842168:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3344 00000100:00080000:2.0:1787828976.842171:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828976.842174:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828976.842199:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828976.842210:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476895872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828976.842211:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476896000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828976.842225:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828976.842228:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828976.842230:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828976.842233:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3344 00000100:00080000:2.0:1787828976.842236:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828976.842238:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828976.842241:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828976.842262:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476896128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828976.842269:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828976.842272:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787828976.842273:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828976.842276:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3344 00000100:00080000:2.0:1787828976.842296:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828976.842299:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828976.842303:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787828976.842313:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476896256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828976.842324:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787828976.842334:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828976.842337:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787828976.842339:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828976.842341:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3344 00000100:00080000:2.0:1787828976.842343:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828976.842346:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787828976.842349:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787828976.842356:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476896384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828976.842408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828976/real 1787828976] req@ffff9a78841d8780 x1874671476895872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828976.842432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476895872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828976.842455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828976.842459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828976.842463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476895872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828976.842499:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828976.842518:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828976.842528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828976/real 1787828976] req@ffff9a78841d9a40 x1874671476896000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828976.842536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476896000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828976.842542:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828976.842545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828976.842547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828976.842551:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828976.842562:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828976.842568:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828976/real 1787828976] req@ffff9a78841da1c0 x1874671476896128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828976.842576:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476896128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828976.842586:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828976.842588:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828976.842590:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828976.842594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828976.842606:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828976.842612:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828976/real 1787828976] req@ffff9a78841db0c0 x1874671476896256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828976.842619:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476896256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828976.842626:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828976.842628:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828976.842630:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828976.842634:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828976.842644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828976.842650:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828976/real 1787828976] req@ffff9a78841dbc00 x1874671476896384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828976.842665:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476896384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828976.842683:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828976.842685:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828976.842688:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787828977.866058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828978.889972:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828979.914000:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828980.937961:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828981.961994:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828981.962034:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828981.962043:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828981.962048:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828981.962060:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3349 00000100:00080000:1.0:1787828981.962072:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828981.962102:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828981.962143:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476896512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828981.962207:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828981.962230:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:3.0:1787828981.962232:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476896512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828981.962233:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828981.962240:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3349 00000100:00080000:1.0:1787828981.962252:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828981.962258:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828981.962277:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828981.962296:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476896640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828981.962306:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828981.962309:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828981.962311:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828981.962314:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3349 00000100:00080000:1.0:1787828981.962320:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828981.962323:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828981.962326:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828981.962334:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787828981.962340:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476896768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828981.962349:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828981.962352:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828981.962356:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828981.962359:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3349 00000100:00080000:1.0:1787828981.962365:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828981.962368:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828981.962372:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828981.962383:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476896896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828981.962391:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828981.962395:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828981.962397:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828981.962402:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3349 00000100:00080000:1.0:1787828981.962422:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828981.962424:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828981/real 1787828981] req@ffff9a7890b1fc00 x1874671476896512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828981.962430:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828981.962445:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828981.962464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476896512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828981.962465:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476897024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828981.962536:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828981.962543:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828981.962547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476896512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828981.962577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828981.962597:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828981.962611:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828981/real 1787828981] req@ffff9a7890b1c000 x1874671476896640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828981.962625:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476896640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828981.962635:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828981.962639:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828981.962643:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828981.962651:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787828981.962669:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828981.962679:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828981/real 1787828981] req@ffff9a7890b1f0c0 x1874671476896768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828981.962687:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476896768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828981.962693:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828981.962695:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828981.962698:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828981.962702:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828981.962714:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828981.962721:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828981/real 1787828981] req@ffff9a7890b1f480 x1874671476896896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828981.962728:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476896896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828981.962735:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828981.962737:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828981.962739:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476896896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828981.962750:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787828981.962762:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828981.962768:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828981/real 1787828981] req@ffff9a7890b1f840 x1874671476897024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828981.962776:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476897024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829006 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828981.962784:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828981.962787:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787828981.962789:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828982.985977:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828984.010030:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828985.033961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828986.057955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828987.081909:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828987.081917:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828987.081921:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828987.081930:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3354 00000100:00080000:1.0:1787828987.081939:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828987.081945:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787828987.081979:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476897152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828987.081983:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828987.082009:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828987.082013:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828987.082015:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828987.082020:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3354 00000100:00080000:1.0:1787828987.082024:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828987.082028:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828987.082039:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828987.082060:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476897280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828987.082065:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476897152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787828987.082072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828987.082075:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828987.082077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828987.082081:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3354 00000100:00080000:1.0:1787828987.082083:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828987.082087:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828987.082091:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828987.082099:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476897408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828987.082106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828987.082109:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787828987.082110:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828987.082113:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3354 00000100:00080000:1.0:1787828987.082115:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828987.082117:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828987.082120:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787828987.082127:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476897536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828987.082133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828987.082136:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787828987.082137:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828987.082140:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3354 00000100:00080000:1.0:1787828987.082143:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828987.082145:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787828987.082148:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787828987.082155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476897664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828987.082197:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828987.082229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787828987.082256:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787828987.082280:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828988.106004:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828989.130027:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828990.154082:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787828990.154139:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787828990.154203:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787828990.154218:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828990.154249:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828990.154259:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828990.154272:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787828990.154286:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787828990.154288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828990] req@ffff9a7890b1f480 x1874671476897280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828990.154319:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476897280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828990.154364:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828990.154371:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828990.154376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828990.154388:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828990] req@ffff9a7890b1f0c0 x1874671476897408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828990.154404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476897408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828990.154413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828990.154418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828990.154421:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828990.154429:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828990] req@ffff9a7890b1c000 x1874671476897536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828990.154442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476897536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828990.154452:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828990.154455:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828990.154459:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828990.154467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828990] req@ffff9a7890b1fc00 x1874671476897664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828990.154479:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476897664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828990.154489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828990.154492:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828990.154495:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897664:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828990.154502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828987/real 1787828990] req@ffff9a7890b1f840 x1874671476897152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828990.154514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476897152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828990.154522:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828990.154526:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828990.154530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476897152:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787828991.178018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828992.201959:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828992.201970:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828992.201975:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828992.201986:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3359 00000100:00080000:3.0:1787828992.201998:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828992.202007:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828992.202043:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476897792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828992.202071:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828992.202085:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828992.202088:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828992.202093:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3359 00000100:00080000:3.0:1787828992.202098:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828992.202103:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787828992.202117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476897792:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787828992.202128:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828992.202148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476897920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828992.202161:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828992.202165:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828992.202167:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828992.202173:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3359 00000800:00000100:2.0:1787828992.202176:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828992.202181:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828992.202188:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828992.202195:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828992.202215:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff9a78a6775680 x1874671476897792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828992.202227:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476898048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828992.202232:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476897792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828992.202255:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828992.202259:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828992.202261:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828992.202267:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828992.202270:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787828992.202275:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3359 00000100:00100000:2.0:1787828992.202279:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476897792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787828992.202282:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828992.202291:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828992.202297:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828992.202312:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476898176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828992.202314:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897920:192.168.203.140@tcp:38:kworker.0 00000100:00080000:3.0:1787828992.202345:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828992.202349:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828992.202351:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828992.202356:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3359 00000100:00080000:3.0:1787828992.202367:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787828992.202369:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787828992.202376:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828992.202382:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787828992.202396:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476898304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828992.202405:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff9a78a6776d00 x1874671476897920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828992.202432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476897920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828992.202441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828992.202445:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828992.202450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476897920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828992.202456:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828992.202480:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828992.202494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff9a78a67761c0 x1874671476898048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828992.202505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476898048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828992.202512:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828992.202515:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828992.202518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828992.202542:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898176:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828992.202563:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:2.0:1787828992.202571:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828992.202586:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff9a78a6774780 x1874671476898176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828992.202597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476898176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828992.202609:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828992.202611:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828992.202613:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828992.202618:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828992.202641:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828992.202654:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828992/real 1787828992] req@ffff9a78a67770c0 x1874671476898304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828992.202665:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476898304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828992.202675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828992.202678:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828992.202681:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787828993.225976:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828994.250018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828995.274009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828996.298007:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828997.321949:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.321957:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828997.321961:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.321969:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3364 00000100:00080000:3.0:1787828997.321977:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.321983:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787828997.322012:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476898432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828997.322037:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.322041:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828997.322042:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.322045:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3364 00000100:00080000:3.0:1787828997.322047:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.322049:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828997.322056:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828997.322065:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476898560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828997.322072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.322074:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828997.322075:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.322078:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3364 00000100:00080000:3.0:1787828997.322080:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.322084:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828997.322087:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828997.322091:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476898432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787828997.322097:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476898688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828997.322105:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.322107:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787828997.322108:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.322111:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3364 00000100:00080000:3.0:1787828997.322112:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.322114:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828997.322118:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787828997.322127:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476898816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828997.322132:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828997.322134:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787828997.322136:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828997.322138:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3364 00000100:00080000:3.0:1787828997.322148:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828997.322151:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787828997.322155:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787828997.322160:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787828997.322167:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476898944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828997.322229:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff9a78a6776940 x1874671476898432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828997.322252:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476898432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828997.322259:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828997.322274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828997.322280:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828997.322285:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476898432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787828997.322293:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828997.322333:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828997.322347:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff9a78a6776580 x1874671476898560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828997.322361:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476898560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828997.322370:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828997.322374:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828997.322378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828997.322404:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787828997.322425:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828997.322437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff9a78a6774b40 x1874671476898688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828997.322451:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476898688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828997.322460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828997.322464:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828997.322467:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787828997.322474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828997.322493:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828997.322504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff9a78a67752c0 x1874671476898816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828997.322517:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476898816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828997.322527:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828997.322531:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828997.322535:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787828997.322541:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787828997.322559:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787828997.322570:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828997/real 1787828997] req@ffff9a78a6775a40 x1874671476898944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828997.322583:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476898944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828997.322594:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828997.322598:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787828997.322601:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476898944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787828998.345965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828999.369989:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829000.393957:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829001.418028:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829002.441976:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829002.441989:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829002.441994:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.442007:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3369 00000100:00080000:3.0:1787829002.442021:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829002.442030:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829002.442074:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476899072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829002.442102:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829002.442105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829002.442107:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.442111:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3369 00000100:00080000:3.0:1787829002.442116:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829002.442121:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829002.442135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829002.442151:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476899200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829002.442163:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829002.442170:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829002.442173:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476899072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829002.442173:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.442183:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3369 00000100:00080000:3.0:1787829002.442190:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829002.442195:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829002.442201:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829002.442219:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476899328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829002.442229:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829002.442233:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829002.442236:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.442240:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3369 00000100:00080000:3.0:1787829002.442254:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829002.442278:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787829002.442282:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787829002.442287:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829002.442305:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476899456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829002.442322:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829002.442325:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829002.442328:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829002.442334:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3369 00000100:00080000:3.0:1787829002.442339:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829002.442341:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff9a78a6774000 x1874671476899072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829002.442351:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829002.442368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476899072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1787829002.442371:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787829002.442390:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829002.442394:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829002.442403:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476899072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829002.442404:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476899584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829002.442440:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829002.442465:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829002.442483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff9a78a6777480 x1874671476899200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829002.442505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476899200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829002.442518:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829002.442523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829002.442528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829002.442537:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899328:192.168.203.140@tcp:38:kworker.0 00010000:00010000:0.0:1787829002.442539:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:2.0:1787829002.442563:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829002.442577:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff9a78a6775a40 x1874671476899328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829002.442597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476899328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829002.442606:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829002.442609:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829002.442611:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829002.442616:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829002.442633:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829002.442641:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff9a78a67752c0 x1874671476899456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829002.442648:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476899456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829002.442655:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829002.442657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829002.442659:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829002.442663:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829002.442674:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829002.442679:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829002/real 1787829002] req@ffff9a78a6774b40 x1874671476899584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829002.442686:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476899584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829002.442696:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829002.442698:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829002.442701:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829003.465991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829004.489944:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829005.514003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829006.537958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829007.561919:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829007.561929:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829007.561933:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829007.561944:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3375 00000100:00080000:1.0:1787829007.561955:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829007.561962:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829007.561978:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829007.561997:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476899712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829007.562022:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829007.562029:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829007.562031:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829007.562035:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3375 00000100:00080000:1.0:1787829007.562038:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829007.562043:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829007.562053:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829007.562064:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476899712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829007.562067:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476899840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829007.562082:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829007.562085:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829007.562087:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829007.562092:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3375 00000100:00080000:1.0:1787829007.562096:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829007.562099:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829007.562104:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829007.562116:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476899968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829007.562123:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829007.562133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829007.562137:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829007.562139:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829007.562144:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3375 00000100:00080000:1.0:1787829007.562148:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829007.562152:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829007.562159:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829007.562168:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9a7890b1e580 x1874671476899712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829007.562174:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476900096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829007.562190:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829007.562191:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476899712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829007.562198:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829007.562203:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829007.562209:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829007.562211:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3375 00000100:00080000:2.0:1787829007.562216:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787829007.562217:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829007.562221:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476899712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829007.562237:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829007.562244:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829007.562256:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476900224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829007.562259:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829007.562286:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829007.562301:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9a7890b1d680 x1874671476899840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829007.562314:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476899840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829007.562324:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829007.562327:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829007.562331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829007.562338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829007.562360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829007.562373:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9a7890b1de00 x1874671476899968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829007.562386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476899968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829007.562395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829007.562398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829007.562402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476899968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829007.562408:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829007.562428:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829007.562441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9a7890b1e940 x1874671476900096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829007.562460:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476900096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829007.562467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829007.562470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829007.562472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829007.562485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829007.562499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829007.562507:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9a7890b1da40 x1874671476900224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829007.562515:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476900224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829007.562521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829007.562523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829007.562525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829008.585967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829009.609983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829010.633996:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829011.658007:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829012.681905:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829012.681913:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829012.681917:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829012.681925:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3380 00000100:00080000:3.0:1787829012.681934:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829012.681939:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829012.681969:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476900352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829012.681995:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829012.681997:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829012.681999:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829012.682001:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3380 00000100:00080000:3.0:1787829012.682003:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829012.682006:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829012.682015:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829012.682023:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476900480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829012.682030:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829012.682032:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829012.682033:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829012.682036:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3380 00000100:00080000:3.0:1787829012.682038:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829012.682042:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829012.682044:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476900352:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787829012.682046:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829012.682059:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476900608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829012.682066:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829012.682068:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829012.682069:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829012.682072:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3380 00000100:00080000:3.0:1787829012.682075:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829012.682077:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829012.682081:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829012.682088:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476900736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829012.682093:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829012.682095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829012.682097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829012.682100:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3380 00000800:00000100:2.0:1787829012.682105:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787829012.682106:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829012.682111:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829012.682114:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829012.682122:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476900864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829012.682158:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829012/real 1787829012] req@ffff9a78a6776580 x1874671476900352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829012.682175:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476900352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829012.682184:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829012.682188:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829012.682203:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476900352:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787829012.682208:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829012.682213:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829012.682233:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829012.682242:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829012/real 1787829012] req@ffff9a78a6776940 x1874671476900480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829012.682249:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476900480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829012.682259:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829012.682262:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829012.682264:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829012.682282:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829012.682293:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829012.682299:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829012/real 1787829012] req@ffff9a78a67770c0 x1874671476900608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829012.682306:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476900608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829012.682311:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829012.682313:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829012.682315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829012.682318:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829012.682328:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829012.682334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829012/real 1787829012] req@ffff9a78a6774780 x1874671476900736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829012.682341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476900736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829012.682346:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829012.682348:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829012.682351:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829012.682354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829012.682363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829012.682368:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829012/real 1787829012] req@ffff9a78a67761c0 x1874671476900864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829012.682375:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476900864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829012.682381:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829012.682382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829012.682384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476900864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829013.705990:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829014.730009:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829015.754008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829016.777938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829017.801972:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829017.801984:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829017.801989:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829017.802000:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3385 00000100:00080000:1.0:1787829017.802013:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829017.802021:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829017.802074:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829017.802077:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476900992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829017.802106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829017.802109:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829017.802111:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829017.802114:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3385 00000100:00080000:1.0:1787829017.802117:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829017.802120:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829017.802130:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829017.802141:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476901120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829017.802150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829017.802153:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829017.802169:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829017.802176:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3385 00000100:00080000:1.0:1787829017.802183:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829017.802183:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476900992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829017.802203:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829017.802216:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829017.802232:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476901248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829017.802240:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829017.802243:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829017.802245:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829017.802249:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3385 00000100:00080000:1.0:1787829017.802268:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829017.802271:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829017.802275:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829017.802278:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829017.802288:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476901376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829017.802297:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829017.802300:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829017.802302:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829017.802305:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3385 00000100:00080000:1.0:1787829017.802309:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829017.802311:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829017.802317:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829017.802325:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476901504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829017.802358:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829017/real 1787829017] req@ffff9a7890b1da40 x1874671476900992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829017.802386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476900992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829017.802395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829017.802398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829017.802401:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476900992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829017.802422:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829017.802454:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829017.802464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829017/real 1787829017] req@ffff9a7890b1e940 x1874671476901120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829017.802472:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476901120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829017.802478:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829017.802481:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829017.802483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829017.802487:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829017.802497:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829017.802511:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829017/real 1787829017] req@ffff9a7890b1de00 x1874671476901248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829017.802518:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476901248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829017.802523:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829017.802525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829017.802527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829017.802531:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829017.802542:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829017.802548:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829017/real 1787829017] req@ffff9a7890b1d680 x1874671476901376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829017.802580:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476901376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829017.802589:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829017.802591:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829017.802594:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829017.802599:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829017.802615:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829017.802638:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829017/real 1787829017] req@ffff9a7890b1e580 x1874671476901504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829017.802646:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476901504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829017.802652:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829017.802654:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829017.802656:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829018.825976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829019.850024:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829020.873935:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829021.897948:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829022.921965:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829022.921977:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:0.0:1787829022.921982:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829022.921982:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829022.922014:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3390 00000100:00080000:2.0:1787829022.922027:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829022.922034:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829022.922081:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476901632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829022.922112:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829022.922116:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829022.922130:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829022.922153:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3390 00000100:00080000:2.0:1787829022.922157:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829022.922161:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829022.922171:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476901632:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829022.922172:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829022.922190:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476901760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829022.922201:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829022.922204:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829022.922206:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829022.922211:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3390 00000100:00080000:2.0:1787829022.922215:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829022.922217:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829022.922221:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829022.922230:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476901888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829022.922237:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829022.922240:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829022.922242:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829022.922247:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3390 00000100:00080000:2.0:1787829022.922257:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829022.922258:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829022.922268:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829022.922273:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829022.922284:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476902016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829022.922290:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829022.922292:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829022.922294:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829022.922298:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3390 00000100:00080000:2.0:1787829022.922302:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829022.922304:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829022.922308:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829022.922323:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476902144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829022.922332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829022/real 1787829022] req@ffff9a78841dbc00 x1874671476901632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829022.922353:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476901632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829022.922362:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829022.922367:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829022.922370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476901632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829022.922391:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829022.922408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829022.922417:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829022/real 1787829022] req@ffff9a78841db0c0 x1874671476901760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829022.922424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476901760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829022.922435:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829022.922437:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829022.922440:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829022.922443:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829022.922453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829022.922459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829022/real 1787829022] req@ffff9a78841da1c0 x1874671476901888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829022.922466:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476901888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829022.922471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829022.922473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829022.922474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476901888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829022.922478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829022.922489:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829022.922494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829022/real 1787829022] req@ffff9a78841d9a40 x1874671476902016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829022.922501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476902016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829022.922507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829022.922509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829022.922512:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829022.922515:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829022.922525:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829022.922530:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829022/real 1787829022] req@ffff9a78841d8780 x1874671476902144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829022.922537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476902144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829022.922561:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829022.922563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829022.922565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829023.945948:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829024.969970:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829025.994019:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829027.017995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829028.041997:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829028.042007:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829028.042012:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829028.042022:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3395 00000100:00080000:1.0:1787829028.042032:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829028.042038:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829028.042075:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476902272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787829028.042096:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829028.042114:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829028.042120:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829028.042124:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829028.042129:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3395 00000100:00080000:1.0:1787829028.042142:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829028.042147:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829028.042159:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829028.042159:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476902272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829028.042181:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476902400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829028.042193:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829028.042197:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829028.042199:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829028.042204:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3395 00000100:00080000:1.0:1787829028.042209:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829028.042214:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829028.042218:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829028.042223:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829028.042239:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476902528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829028.042251:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829028.042254:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829028.042256:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829028.042261:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3395 00000100:00080000:1.0:1787829028.042265:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829028.042272:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff9a7890b1f840 x1874671476902272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829028.042273:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829028.042293:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829028.042304:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476902272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829028.042310:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476902656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829028.042315:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829028.042324:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787829028.042331:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829028.042331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476902272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829028.042339:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829028.042342:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829028.042348:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3395 00000100:00080000:1.0:1787829028.042353:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829028.042355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902400:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829028.042361:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829028.042369:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787829028.042375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829028.042386:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476902784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829028.042389:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff9a7890b1fc00 x1874671476902400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829028.042420:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476902400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829028.042427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829028.042430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829028.042432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829028.042437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829028.042449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829028.042455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff9a7890b1c000 x1874671476902528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829028.042462:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476902528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829028.042467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829028.042469:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829028.042471:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829028.042474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829028.042484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829028.042489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff9a7890b1f0c0 x1874671476902656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829028.042496:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476902656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829028.042503:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829028.042505:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829028.042506:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829028.042516:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829028.042525:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829028.042531:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829028/real 1787829028] req@ffff9a7890b1f480 x1874671476902784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829028.042537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476902784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829028.042544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829028.042546:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829028.042547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476902784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829029.066009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829030.089983:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829031.114021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829032.137992:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829033.161915:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829033.161922:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829033.161927:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829033.161935:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3400 00000100:00080000:1.0:1787829033.161944:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829033.161950:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829033.161974:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829033.161989:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476902912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829033.162015:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829033.162019:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829033.162021:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829033.162024:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3400 00000100:00080000:1.0:1787829033.162029:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829033.162034:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829033.162043:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829033.162066:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476902912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829033.162070:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476903040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829033.162081:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829033.162083:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829033.162085:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829033.162088:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3400 00000100:00080000:1.0:1787829033.162091:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829033.162094:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829033.162097:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829033.162106:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476903168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829033.162113:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829033.162115:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829033.162116:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829033.162119:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3400 00000100:00080000:1.0:1787829033.162122:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829033.162124:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829033.162128:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829033.162136:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829033.162137:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476903296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829033.162146:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829033.162149:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829033.162150:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829033.162154:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3400 00000100:00080000:1.0:1787829033.162163:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829033.162165:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829033.162168:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829033.162175:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476903424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829033.162195:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff9a7890b1f480 x1874671476902912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829033.162212:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476902912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829033.162220:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829033.162223:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829033.162226:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476902912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829033.162246:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829033.162270:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829033.162280:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff9a7890b1f0c0 x1874671476903040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829033.162289:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476903040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829033.162294:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829033.162297:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829033.162299:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829033.162303:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829033.162315:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829033.162320:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff9a7890b1c000 x1874671476903168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829033.162328:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476903168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829033.162333:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829033.162334:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829033.162336:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829033.162340:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829033.162349:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829033.162355:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff9a7890b1fc00 x1874671476903296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829033.162362:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476903296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829033.162368:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829033.162370:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829033.162372:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829033.162375:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829033.162384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829033.162390:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829033/real 1787829033] req@ffff9a7890b1f840 x1874671476903424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829033.162397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476903424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829033.162405:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829033.162407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829033.162410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829034.185996:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829035.209987:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829036.233977:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829037.258090:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829038.281975:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.281984:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829038.281988:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829038.281998:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3405 00000100:00080000:1.0:1787829038.282009:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829038.282032:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829038.282038:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829038.282065:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476903552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829038.282088:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.282091:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829038.282092:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829038.282095:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3405 00000100:00080000:1.0:1787829038.282098:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829038.282102:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829038.282109:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829038.282117:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476903680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829038.282125:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.282128:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829038.282129:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829038.282133:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3405 00000100:00080000:1.0:1787829038.282138:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829038.282139:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476903552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829038.282144:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829038.282158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829038.282169:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476903808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829038.282176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.282179:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829038.282180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829038.282183:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3405 00000100:00080000:1.0:1787829038.282186:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829038.282189:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829038.282193:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829038.282207:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476903936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829038.282217:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829038.282221:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829038.282224:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829038.282225:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829038.282236:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3405 00000100:00080000:1.0:1787829038.282243:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829038.282249:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829038.282254:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829038.282266:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476904064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829038.282314:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9a7890b1e580 x1874671476903552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.282334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476903552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.282348:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.282353:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829038.282359:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476903552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829038.282389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829038.282410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829038.282420:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9a7890b1d680 x1874671476903680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.282429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476903680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.282436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.282438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829038.282441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829038.282446:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829038.282457:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829038.282463:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9a7890b1de00 x1874671476903808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.282471:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476903808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.282477:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.282479:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829038.282481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829038.282485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829038.282495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829038.282501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9a7890b1e940 x1874671476903936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.282507:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476903936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.282521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.282524:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829038.282526:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476903936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829038.282530:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829038.282543:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829038.282549:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9a7890b1da40 x1874671476904064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829038.282557:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476904064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829038.282564:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829038.282565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829038.282567:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829039.305985:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829040.330103:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829041.353989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829042.378087:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829043.401911:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829043.401923:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829043.401927:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829043.401938:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3410 00000100:00080000:1.0:1787829043.401949:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829043.401957:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829043.402006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829043.402030:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476904192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829043.402053:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829043.402056:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829043.402058:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829043.402061:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3410 00000100:00080000:1.0:1787829043.402064:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829043.402068:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829043.402078:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829043.402088:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476904192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829043.402090:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476904320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829043.402104:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829043.402108:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829043.402110:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829043.402114:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3410 00000100:00080000:1.0:1787829043.402116:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829043.402118:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829043.402121:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829043.402130:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476904448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829043.402137:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829043.402140:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829043.402141:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829043.402144:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3410 00000100:00080000:1.0:1787829043.402158:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829043.402161:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829043.402164:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829043.402175:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829043.402178:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476904576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829043.402186:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829043.402189:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829043.402191:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829043.402193:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3410 00000100:00080000:1.0:1787829043.402197:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829043.402200:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829043.402204:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829043.402211:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476904704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829043.402243:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9a7890b1da40 x1874671476904192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829043.402260:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476904192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829043.402269:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829043.402273:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829043.402276:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476904192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829043.402300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829043.402317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829043.402326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9a7890b1e940 x1874671476904320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829043.402335:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476904320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829043.402340:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829043.402342:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829043.402344:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829043.402347:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829043.402359:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829043.402364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9a7890b1de00 x1874671476904448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829043.402372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476904448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829043.402382:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829043.402384:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829043.402386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829043.402389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829043.402399:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829043.402405:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9a7890b1d680 x1874671476904576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829043.402411:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476904576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829043.402418:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829043.402420:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829043.402422:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829043.402425:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829043.402434:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829043.402439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9a7890b1e580 x1874671476904704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829043.402446:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476904704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829043.402453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829043.402454:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829043.402457:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829044.426022:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829045.450045:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829046.473992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829047.498088:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829048.521923:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829048.521934:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829048.521940:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829048.521965:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3416 00000100:00080000:2.0:1787829048.521977:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829048.521985:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829048.521993:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829048.522031:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476904832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829048.522054:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829048.522057:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829048.522059:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829048.522062:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3416 00000100:00080000:2.0:1787829048.522064:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829048.522067:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829048.522079:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829048.522093:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476904832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829048.522096:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476904960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829048.522111:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829048.522116:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829048.522118:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829048.522123:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3416 00000100:00080000:2.0:1787829048.522128:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829048.522133:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829048.522139:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829048.522153:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476905088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829048.522171:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829048.522180:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829048.522183:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829048.522185:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829048.522191:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3416 00000100:00080000:2.0:1787829048.522202:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829048.522206:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829048.522212:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829048.522228:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476905216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829048.522239:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9a78841d9e00 x1874671476904832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829048.522241:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829048.522258:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829048.522260:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829048.522265:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3416 00000100:00080000:2.0:1787829048.522270:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829048.522270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476904832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829048.522279:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829048.522289:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829048.522290:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829048.522299:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829048.522305:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476904832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829048.522308:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476905344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829048.522341:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829048.522370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829048.522385:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9a78841da580 x1874671476904960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829048.522400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476904960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829048.522409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829048.522413:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829048.522417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476904960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829048.522424:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829048.522444:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829048.522621:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9a78841d8000 x1874671476905088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829048.522633:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476905088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829048.522640:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829048.522643:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829048.522645:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829048.522651:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829048.522671:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829048.522684:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9a78841d83c0 x1874671476905216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829048.522697:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476905216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829048.522709:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829048.522713:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829048.522716:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829048.522723:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829048.522742:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829048.522753:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9a78841db480 x1874671476905344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829048.522761:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476905344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829048.522769:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829048.522771:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829048.522774:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829049.546038:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829050.570008:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829051.593949:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829052.617971:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829053.641954:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829053.641965:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829053.641969:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829053.641980:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3421 00000100:00080000:2.0:1787829053.641991:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829053.641999:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829053.641999:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829053.642036:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476905472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829053.642056:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829053.642058:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829053.642060:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829053.642062:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3421 00000100:00080000:2.0:1787829053.642066:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829053.642069:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829053.642077:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829053.642084:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476905472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829053.642089:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476905600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829053.642097:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829053.642099:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829053.642101:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829053.642104:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3421 00000100:00080000:2.0:1787829053.642106:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829053.642108:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829053.642112:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829053.642120:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476905728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829053.642127:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829053.642130:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829053.642131:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829053.642134:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3421 00000100:00080000:2.0:1787829053.642137:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829053.642139:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829053.642143:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829053.642152:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476905856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829053.642158:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829053.642161:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829053.642162:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829053.642166:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3421 00000100:00080000:2.0:1787829053.642168:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829053.642171:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829053.642174:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829053.642181:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476905984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829053.642187:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829053.642216:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829053.642229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829053.642244:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829054.666074:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829055.690008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829056.714013:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829056.714080:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787829056.714116:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829056.714133:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829056.714172:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829056.714183:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829056.714192:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829056.714199:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829056.714212:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829056] req@ffff9a78841db840 x1874671476905600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829056.714237:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476905600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829056.714249:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829056.714255:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829056.714260:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829056.714269:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829056] req@ffff9a78841d92c0 x1874671476905728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829056.714290:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476905728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829056.714305:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829056.714309:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829056.714313:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829056.714320:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829056] req@ffff9a78841d9680 x1874671476905856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829056.714334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476905856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829056.714344:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829056.714347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829056.714350:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829056.714357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829056] req@ffff9a78841d8b40 x1874671476905472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829056.714370:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476905472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829056.714378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829056.714382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829056.714385:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476905472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829056.714412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829056] req@ffff9a78841da940 x1874671476905984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829056.714426:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476905984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829056.714437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829056.714441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829056.714444:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476905984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829057.737996:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829058.761967:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829058.761988:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829058.761996:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829058.762000:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829058.762010:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3426 00000100:00080000:1.0:1787829058.762021:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829058.762046:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829058.762080:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476906112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829058.762106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829058.762110:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829058.762112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829058.762117:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3426 00000100:00080000:1.0:1787829058.762121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829058.762125:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829058.762136:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829058.762151:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476906240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829058.762163:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829058.762166:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476906112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829058.762167:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829058.762171:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829058.762191:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3426 00000100:00080000:1.0:1787829058.762196:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829058.762200:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829058.762205:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829058.762217:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476906368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829058.762228:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829058.762232:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829058.762234:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829058.762240:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3426 00000800:00000100:3.0:1787829058.762245:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829058.762248:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829058.762254:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829058.762260:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829058.762272:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476906496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829058.762283:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829058.762287:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829058.762289:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829058.762294:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3426 00000100:00100000:3.0:1787829058.762300:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829058] req@ffff9a7890b1f840 x1874671476906112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829058.762301:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829058.762308:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829058.762317:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829058.762331:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476906624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829058.762336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476906112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829058.762359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829058.762364:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829058.762370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476906112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829058.762398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829058.762425:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829058.762440:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829058] req@ffff9a7890b1fc00 x1874671476906240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829058.762450:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476906240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829058.762460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829058.762463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829058.762466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829058.762473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829058.762493:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829058.762506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829058] req@ffff9a7890b1c000 x1874671476906368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829058.762526:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476906368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829058.762535:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829058.762538:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829058.762542:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829058.762549:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829058.762570:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829058.762581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829058] req@ffff9a7890b1f0c0 x1874671476906496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829058.762592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476906496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829058.762602:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829058.762605:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829058.762608:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829058.762614:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829058.762632:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829058.762644:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829058] req@ffff9a7890b1f480 x1874671476906624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829058.762655:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476906624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829058.762664:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829058.762667:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829058.762670:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829059.786001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829060.810040:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829061.833965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829062.857954:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829063.881955:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829063.881965:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829063.881969:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829063.881979:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3431 00000100:00080000:0.0:1787829063.881991:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829063.881998:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829063.882006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829063.882033:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476906752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829063.882058:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829063.882065:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829063.882067:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829063.882071:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3431 00000100:00080000:0.0:1787829063.882075:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829063.882080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829063.882089:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829063.882096:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476906752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829063.882106:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476906880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829063.882119:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829063.882122:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829063.882124:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829063.882129:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3431 00000100:00080000:0.0:1787829063.882149:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829063.882153:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829063.882157:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829063.882161:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829063.882173:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476907008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829063.882186:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829063.882197:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829063.882200:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829063.882205:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3431 00000100:00080000:0.0:1787829063.882210:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829063.882215:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9a7890a3f480 x1874671476906752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829063.882217:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829063.882226:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829063.882244:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476906752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829063.882245:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476907136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829063.882279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829063.882285:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829063.882288:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787829063.882291:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829063.882295:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829063.882297:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476906752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829063.882315:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3431 00000100:00080000:0.0:1787829063.882320:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829063.882324:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829063.882328:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829063.882338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787829063.882343:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476907264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829063.882378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829063.882393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9a7890a3de00 x1874671476906880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829063.882407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476906880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829063.882422:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829063.882426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829063.882431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476906880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829063.882437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829063.882456:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829063.882468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9a7890a3fc00 x1874671476907008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829063.882481:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476907008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829063.882489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829063.882493:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829063.882496:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829063.882503:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829063.882520:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829063.882531:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9a7890a3c3c0 x1874671476907136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829063.882543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476907136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829063.882552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829063.882555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829063.882559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829063.882575:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829063.882597:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829063.882609:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9a7890a3da40 x1874671476907264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829063.882622:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476907264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829063.882633:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829063.882636:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829063.882639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829064.906010:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829065.930037:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829066.954024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829067.978104:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829069.001952:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829069.001962:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829069.001967:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829069.001977:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3436 00010000:00010000:2.0:1787829069.001988:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829069.001989:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829069.001998:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829069.002046:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476907392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829069.002072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829069.002077:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829069.002079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829069.002083:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3436 00000100:00080000:1.0:1787829069.002087:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829069.002091:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829069.002101:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829069.002115:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476907520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829069.002115:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476907392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829069.002131:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829069.002134:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829069.002136:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829069.002139:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3436 00000100:00080000:1.0:1787829069.002142:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829069.002145:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829069.002149:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829069.002158:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476907648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829069.002166:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829069.002170:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787829069.002171:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829069.002172:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829069.002175:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3436 00000100:00080000:1.0:1787829069.002178:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829069.002182:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829069.002186:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829069.002195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476907776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829069.002203:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829069.002206:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829069.002207:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829069.002211:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3436 00000100:00080000:1.0:1787829069.002226:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829069.002229:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829069.002229:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff9a7890b1f480 x1874671476907392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829069.002233:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829069.002246:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476907392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829069.002251:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476907904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829069.002257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829069.002261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829069.002266:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476907392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829069.002301:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829069.002321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829069.002332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff9a7890b1f0c0 x1874671476907520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829069.002340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476907520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829069.002346:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829069.002348:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829069.002350:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829069.002354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829069.002364:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829069.002370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff9a7890b1c000 x1874671476907648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829069.002377:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476907648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829069.002386:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829069.002389:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829069.002392:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829069.002398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829069.002418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829069.002430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff9a7890b1fc00 x1874671476907776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829069.002443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476907776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829069.002451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829069.002453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829069.002455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829069.002459:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829069.002473:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829069.002480:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829069/real 1787829069] req@ffff9a7890b1f840 x1874671476907904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829069.002487:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476907904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829069.002493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829069.002495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829069.002496:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476907904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829070.025959:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829071.050067:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829072.074030:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829073.098134:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829074.121940:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829074.121950:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829074.121955:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829074.121965:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3441 00000100:00080000:1.0:1787829074.121977:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829074.121984:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829074.121992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829074.122018:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476908032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829074.122045:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829074.122050:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829074.122052:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829074.122057:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3441 00000100:00080000:1.0:1787829074.122062:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829074.122066:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829074.122075:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829074.122079:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476908032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829074.122107:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476908160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829074.122118:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829074.122123:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829074.122125:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829074.122129:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3441 00000100:00080000:1.0:1787829074.122132:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829074.122135:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829074.122139:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829074.122150:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476908288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829074.122158:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787829074.122164:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829074.122166:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829074.122171:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829074.122177:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3441 00000100:00080000:1.0:1787829074.122183:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829074.122188:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829074.122195:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829074.122211:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476908416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829074.122223:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829074.122225:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829074] req@ffff9a7890b1e580 x1874671476908032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829074.122231:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829074.122236:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829074.122247:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3441 00000100:00080000:1.0:1787829074.122253:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829074.122256:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476908032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829074.122263:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829074.122274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787829074.122276:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829074.122281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829074.122287:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476908032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829074.122295:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476908544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829074.122314:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829074.122339:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829074.122351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829074] req@ffff9a7890b1d680 x1874671476908160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829074.122360:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476908160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829074.122376:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829074.122379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829074.122381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829074.122386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829074.122400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829074.122408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829074] req@ffff9a7890b1de00 x1874671476908288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829074.122433:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476908288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829074.122439:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829074.122442:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829074.122445:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829074.122451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829074.122473:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829074.122485:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829074] req@ffff9a7890b1e940 x1874671476908416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829074.122498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476908416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829074.122507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829074.122509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829074.122512:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829074.122516:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829074.122531:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829074.122538:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829074/real 1787829074] req@ffff9a7890b1da40 x1874671476908544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829074.122546:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476908544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829074.122552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829074.122554:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829074.122556:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829075.145975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829076.169957:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829077.194009:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829078.217965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829079.241928:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.241935:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829079.241939:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829079.241949:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3446 00000100:00080000:1.0:1787829079.241961:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829079.241968:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829079.241975:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829079.242007:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476908672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829079.242029:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.242033:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829079.242036:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829079.242041:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3446 00000100:00080000:1.0:1787829079.242046:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829079.242051:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829079.242060:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476908672:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787829079.242061:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829079.242080:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476908800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829079.242092:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.242096:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829079.242098:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829079.242102:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3446 00000100:00080000:1.0:1787829079.242108:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829079.242112:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829079.242117:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829079.242122:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829079.242135:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476908928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829079.242150:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.242153:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829079.242154:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829079.242158:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3446 00000100:00080000:1.0:1787829079.242161:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829079.242164:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829079.242169:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829079.242172:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9a7890b1da40 x1874671476908672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829079.242183:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476909056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829079.242188:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476908672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.242198:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.242204:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829079.242211:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476908672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829079.242218:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829079.242221:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829079.242223:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829079.242227:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3446 00000100:00080000:1.0:1787829079.242230:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829079.242233:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908800:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829079.242237:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829079.242242:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829079.242253:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476909184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829079.242269:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829079.242288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9a7890b1e940 x1874671476908800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829079.242303:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476908800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.242312:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.242314:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829079.242317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829079.242321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829079.242337:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829079.242345:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9a7890b1de00 x1874671476908928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829079.242352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476908928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.242358:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.242360:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829079.242363:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476908928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829079.242366:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829079.242377:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829079.242382:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9a7890b1d680 x1874671476909056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829079.242389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476909056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.242396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.242398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829079.242400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829079.242419:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829079.242429:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829079.242435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9a7890b1e580 x1874671476909184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829079.242442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476909184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829079.242448:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829079.242450:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829079.242451:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829080.266019:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829081.290028:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829082.314025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829083.337990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829084.361912:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829084.361921:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829084.361924:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829084.361932:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3451 00000100:00080000:2.0:1787829084.361941:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829084.361944:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829084.361948:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829084.361980:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476909312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829084.362000:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829084.362002:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829084.362004:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829084.362007:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3451 00000100:00080000:2.0:1787829084.362009:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829084.362012:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829084.362020:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829084.362031:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476909312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829084.362032:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476909440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829084.362050:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829084.362052:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829084.362054:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829084.362057:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3451 00000100:00080000:2.0:1787829084.362059:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829084.362062:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829084.362065:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829084.362073:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476909568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829084.362078:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829084.362081:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829084.362082:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829084.362085:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3451 00000100:00080000:2.0:1787829084.362087:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829084.362090:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829084.362095:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829084.362096:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829084.362106:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476909696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829084.362123:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829084.362126:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829084.362128:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829084.362131:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3451 00000100:00080000:2.0:1787829084.362135:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829084.362138:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829084.362141:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829084.362149:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476909824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829084.362169:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9a78841da940 x1874671476909312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829084.362185:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476909312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829084.362194:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829084.362197:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829084.362207:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476909312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829084.362231:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829084.362257:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829084.362271:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9a78841d8b40 x1874671476909440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829084.362281:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476909440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829084.362287:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829084.362289:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829084.362291:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829084.362295:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829084.362309:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829084.362315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9a78841d9680 x1874671476909568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829084.362323:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476909568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829084.362333:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829084.362335:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829084.362337:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829084.362340:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829084.362360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829084.362370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9a78841d92c0 x1874671476909696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829084.362377:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476909696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829084.362383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829084.362386:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829084.362388:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829084.362391:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829084.362401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829084.362407:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9a78841db840 x1874671476909824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829084.362418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476909824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829084.362427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829084.362430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829084.362434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476909824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829085.385947:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829086.409958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829087.433952:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829088.457917:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829089.481895:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829089.481903:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829089.481907:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829089.481916:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3456 00000100:00080000:2.0:1787829089.481924:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829089.481930:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829089.481965:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476909952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829089.481968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829089.481995:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829089.481999:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829089.482002:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829089.482007:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3456 00000100:00080000:2.0:1787829089.482012:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829089.482016:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829089.482025:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829089.482028:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476909952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829089.482038:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476910080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829089.482048:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829089.482050:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829089.482052:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829089.482055:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3456 00000100:00080000:2.0:1787829089.482058:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829089.482060:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829089.482063:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829089.482071:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476910208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829089.482077:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829089.482080:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829089.482081:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829089.482083:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3456 00000800:00000100:3.0:1787829089.482087:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829089.482088:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829089.482094:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829089.482098:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829089.482107:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476910336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829089.482113:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829089.482116:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829089.482117:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829089.482120:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3456 00000100:00080000:2.0:1787829089.482123:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829089.482126:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829089.482130:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829089.482133:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9a78841db480 x1874671476909952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829089.482141:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476910464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829089.482152:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476909952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829089.482162:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829089.482165:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829089.482169:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476909952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829089.482187:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829089.482201:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829089.482209:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9a78841d83c0 x1874671476910080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829089.482217:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476910080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829089.482222:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829089.482224:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829089.482227:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829089.482230:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829089.482239:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829089.482245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9a78841d8000 x1874671476910208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829089.482252:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476910208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829089.482257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829089.482259:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829089.482261:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829089.482264:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829089.482274:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829089.482280:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9a78841da580 x1874671476910336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829089.482287:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476910336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829089.482293:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829089.482294:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829089.482296:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829089.482300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829089.482308:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829089.482314:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9a78841d9e00 x1874671476910464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829089.482321:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476910464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829089.482327:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829089.482329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829089.482331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829090.506004:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829091.530024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829092.554002:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829093.577968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829094.601908:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829094.601918:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829094.601923:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829094.601933:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3462 00000100:00080000:2.0:1787829094.601957:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829094.601965:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829094.601966:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829094.602002:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476910592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829094.602027:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829094.602031:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829094.602032:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829094.602036:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3462 00000100:00080000:2.0:1787829094.602040:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829094.602044:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829094.602052:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829094.602053:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476910592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829094.602066:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476910720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829094.602079:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829094.602084:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829094.602086:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829094.602091:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3462 00000100:00080000:2.0:1787829094.602104:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829094.602111:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829094.602112:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829094.602121:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829094.602133:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476910848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829094.602142:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829094.602146:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829094.602148:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829094.602153:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3462 00000100:00080000:2.0:1787829094.602157:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829094.602161:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829094.602167:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829094.602177:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829094/real 1787829094] req@ffff9a78841d8780 x1874671476910592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829094.602184:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476910976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829094.602197:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829094.602198:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476910592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829094.602203:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829094.602206:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829094.602209:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3462 00000100:00080000:2.0:1787829094.602214:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829094.602220:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829094.602221:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:2.0:1787829094.602227:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829094.602228:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829094.602235:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476910592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829094.602242:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476911104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829094.602259:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829094.602286:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829094.602300:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829094/real 1787829094] req@ffff9a78841d9a40 x1874671476910720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829094.602313:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476910720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829094.602319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829094.602321:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829094.602324:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829094.602329:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829094.602344:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829094.602351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829094/real 1787829094] req@ffff9a78841da1c0 x1874671476910848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829094.602359:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476910848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829094.602364:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829094.602366:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829094.602368:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829094.602372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829094.602381:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829094.602387:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829094/real 1787829094] req@ffff9a78841db0c0 x1874671476910976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829094.602394:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476910976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829094.602400:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829094.602402:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829094.602404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476910976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829094.602407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829094.602417:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829094.602423:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829094/real 1787829094] req@ffff9a78841dbc00 x1874671476911104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829094.602430:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476911104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829094.602436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829094.602438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829094.602440:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829095.625983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829096.650000:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829097.674063:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829098.698076:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829099.721961:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829099.721972:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829099.721977:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829099.721988:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3467 00000100:00080000:2.0:1787829099.722000:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829099.722007:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829099.722038:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829099.722055:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476911232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829099.722079:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829099.722082:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829099.722084:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829099.722087:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3467 00000100:00080000:2.0:1787829099.722089:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829099.722092:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829099.722102:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829099.722111:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476911360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829099.722119:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829099.722124:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829099.722126:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829099.722127:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476911232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829099.722130:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3467 00000100:00080000:2.0:1787829099.722135:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829099.722140:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829099.722143:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829099.722150:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476911488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829099.722156:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829099.722159:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829099.722160:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829099.722163:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3467 00000100:00080000:2.0:1787829099.722165:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829099.722168:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829099.722172:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829099.722179:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476911616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829099.722185:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829099.722188:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829099.722189:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829099.722191:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3467 00000100:00080000:2.0:1787829099.722201:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829099.722203:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829099.722206:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829099.722216:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476911744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829099.722216:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829099.722292:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829099/real 1787829099] req@ffff9a78841dbc00 x1874671476911232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829099.722314:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476911232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829099.722328:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829099.722332:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829099.722337:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476911232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829099.722364:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829099.722388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829099.722402:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829099/real 1787829099] req@ffff9a78841db0c0 x1874671476911360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829099.722415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476911360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829099.722430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829099.722434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829099.722437:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829099.722443:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829099.722461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829099.722471:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829099/real 1787829099] req@ffff9a78841da1c0 x1874671476911488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829099.722481:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476911488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829099.722490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829099.722493:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829099.722497:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829099.722503:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829099.722521:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829099.722531:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829099/real 1787829099] req@ffff9a78841d9a40 x1874671476911616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829099.722542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476911616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829099.722551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829099.722555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829099.722559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829099.722565:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829099.722583:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829099.722613:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829099/real 1787829099] req@ffff9a78841d8780 x1874671476911744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829099.722626:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476911744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829099.722637:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829099.722641:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829099.722644:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476911744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829100.746054:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829101.769976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829102.794020:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829103.818016:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829104.841952:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829104.841963:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829104.841969:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829104.841981:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3472 00000100:00080000:1.0:1787829104.841994:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829104.842002:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829104.842020:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829104.842047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476911872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829104.842080:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829104.842083:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829104.842084:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829104.842087:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3472 00000100:00080000:1.0:1787829104.842090:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829104.842094:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829104.842104:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829104.842114:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476912000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829104.842124:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829104.842128:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829104.842128:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476911872:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829104.842132:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829104.842146:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3472 00000100:00080000:1.0:1787829104.842149:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829104.842151:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829104.842155:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829104.842169:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476912128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829104.842177:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829104.842179:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829104.842180:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829104.842183:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3472 00000100:00080000:1.0:1787829104.842185:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829104.842188:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829104.842191:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829104.842197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476912256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829104.842204:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829104.842207:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829104.842209:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829104.842216:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3472 00000800:00000100:3.0:1787829104.842220:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829104.842224:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829104.842235:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829104.842239:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829104.842252:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476912384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829104.842290:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829104/real 1787829104] req@ffff9a7890b1f840 x1874671476911872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829104.842313:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476911872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829104.842326:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829104.842329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829104.842333:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476911872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829104.842355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829104.842374:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829104.842384:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829104/real 1787829104] req@ffff9a7890b1fc00 x1874671476912000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829104.842392:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476912000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829104.842397:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829104.842400:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829104.842402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829104.842406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829104.842416:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829104.842422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829104/real 1787829104] req@ffff9a7890b1c000 x1874671476912128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829104.842430:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476912128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829104.842434:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829104.842436:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829104.842438:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829104.842442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829104.842452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829104.842458:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829104/real 1787829104] req@ffff9a7890b1f0c0 x1874671476912256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829104.842464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476912256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829104.842471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829104.842473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829104.842475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829104.842479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829104.842489:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829104.842495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829104/real 1787829104] req@ffff9a7890b1f480 x1874671476912384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829104.842501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476912384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829104.842507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829104.842509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829104.842511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829105.866073:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829106.889957:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829107.914063:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829108.938090:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829109.961943:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829109.961951:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829109.961954:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829109.961962:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3477 00000100:00080000:0.0:1787829109.961971:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829109.961978:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829109.962006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829109.962011:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476912512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829109.962034:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829109.962036:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829109.962038:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829109.962041:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3477 00000100:00080000:0.0:1787829109.962044:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829109.962046:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829109.962054:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829109.962065:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476912640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829109.962070:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476912512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829109.962077:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829109.962080:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829109.962081:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829109.962084:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3477 00000100:00080000:0.0:1787829109.962086:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829109.962088:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829109.962091:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829109.962101:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476912768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829109.962108:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829109.962111:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829109.962112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829109.962114:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3477 00000100:00080000:0.0:1787829109.962119:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829109.962121:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829109.962124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829109.962128:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829109.962134:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476912896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829109.962142:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829109.962144:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829109.962146:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829109.962149:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3477 00000100:00080000:0.0:1787829109.962150:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829109.962152:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829109.962156:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829109.962164:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476913024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829109.962174:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829109/real 1787829109] req@ffff9a7890a3c000 x1874671476912512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829109.962188:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476912512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829109.962208:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829109.962212:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829109.962215:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476912512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829109.962236:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829109.962260:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829109.962275:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829109/real 1787829109] req@ffff9a7890a3da40 x1874671476912640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829109.962289:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476912640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829109.962298:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829109.962301:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829109.962303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829109.962308:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829109.962330:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829109.962343:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829109/real 1787829109] req@ffff9a7890a3c3c0 x1874671476912768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829109.962356:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476912768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829109.962365:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829109.962368:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829109.962370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829109.962374:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829109.962391:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829109.962399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829109/real 1787829109] req@ffff9a7890a3fc00 x1874671476912896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829109.962407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476912896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829109.962413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829109.962415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829109.962417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476912896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829109.962421:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829109.962432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829109.962438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829109/real 1787829109] req@ffff9a7890a3de00 x1874671476913024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829109.962445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476913024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829109.962451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829109.962453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829109.962454:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829110.986030:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829112.010029:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829113.034039:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829114.057988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829115.082016:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.082032:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829115.082038:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787829115.082045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829115.082051:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3482 00000100:00080000:0.0:1787829115.082065:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.082072:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829115.082122:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476913152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829115.082151:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.082154:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829115.082155:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829115.082159:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3482 00000100:00080000:0.0:1787829115.082162:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.082167:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829115.082178:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829115.082200:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476913152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829115.082201:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476913280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829115.082241:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.082244:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829115.082248:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829115.082253:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3482 00000100:00080000:0.0:1787829115.082272:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.082276:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829115.082281:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829115.082290:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829115.082298:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476913408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829115.082312:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.082315:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829115.082317:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829115.082324:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3482 00000100:00080000:0.0:1787829115.082329:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.082334:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829115.082337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9a7890a3f480 x1874671476913152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787829115.082346:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829115.082355:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476913152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829115.082366:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476913536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829115.082366:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829115.082381:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829115.082393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476913152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829115.082401:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.082406:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829115.082408:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829115.082414:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3482 00000100:00080000:0.0:1787829115.082418:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.082422:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829115.082425:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913280:192.168.203.140@tcp:38:kworker.0 00000008:00100000:0.0:1787829115.082432:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829115.082451:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476913664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829115.082462:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829115.082553:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9a7890a3de00 x1874671476913280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829115.082564:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476913280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829115.082570:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829115.082573:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829115.082575:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829115.082581:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829115.082599:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829115.082607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9a7890a3fc00 x1874671476913408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829115.082614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476913408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829115.082620:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829115.082622:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829115.082625:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829115.082629:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829115.082640:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829115.082646:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9a7890a3c3c0 x1874671476913536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829115.082653:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476913536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829115.082660:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829115.082662:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829115.082663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829115.082675:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829115.082695:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829115.082702:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9a7890a3da40 x1874671476913664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829115.082709:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476913664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829115.082716:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829115.082718:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829115.082720:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829116.106048:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829117.129993:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829118.154039:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829119.178021:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829120.201950:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829120.201966:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829120.201972:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829120.201986:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3487 00010000:00010000:2.0:1787829120.201994:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829120.202002:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829120.202011:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829120.202063:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476913792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829120.202087:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829120.202090:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829120.202092:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829120.202095:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3487 00000100:00080000:1.0:1787829120.202099:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829120.202103:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829120.202114:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829120.202120:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476913792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829120.202129:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476913920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829120.202138:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829120.202141:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829120.202143:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829120.202146:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3487 00000100:00080000:1.0:1787829120.202148:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829120.202152:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829120.202156:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829120.202165:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476914048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829120.202171:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829120.202174:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829120.202175:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829120.202178:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3487 00000100:00080000:1.0:1787829120.202182:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829120.202184:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829120.202188:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829120.202195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476914176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829120.202202:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829120.202204:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829120.202206:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829120.202208:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3487 00000100:00080000:1.0:1787829120.202210:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829120.202212:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829120.202215:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829120.202224:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476914304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829120.202262:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829120.202292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829120.202306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829120.202322:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829121.226031:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829122.250015:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829123.274011:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787829123.274163:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787829123.274224:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787829123.274240:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829123.274290:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829123.274302:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829123.274313:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829123.274325:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787829123.274338:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829123] req@ffff9a7890b1f0c0 x1874671476913920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829123.274366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476913920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829123.274393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829123.274399:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829123.274404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476913920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829123.274412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829123] req@ffff9a7890b1c000 x1874671476914048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829123.274438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476914048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829123.274447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829123.274451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829123.274454:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829123.274461:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829123] req@ffff9a7890b1fc00 x1874671476914176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829123.274473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476914176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829123.274482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829123.274485:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829123.274488:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829123.274494:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829123] req@ffff9a7890b1f840 x1874671476914304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829123.274504:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476914304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829123.274513:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829123.274516:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829123.274519:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914304:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829123.274525:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829120/real 1787829123] req@ffff9a7890b1f480 x1874671476913792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829123.274536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476913792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829123.274544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829123.274547:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829123.274550:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476913792:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787829124.297955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829125.321947:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829125.321959:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829125.321964:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829125.321976:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3492 00000100:00080000:1.0:1787829125.321989:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829125.322000:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829125.322005:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829125.322042:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476914432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829125.322068:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829125.322071:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829125.322072:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829125.322075:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3492 00000100:00080000:1.0:1787829125.322077:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829125.322080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829125.322090:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829125.322101:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476914560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829125.322111:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829125.322113:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829125.322115:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829125.322118:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3492 00000100:00080000:1.0:1787829125.322121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829125.322129:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829125.322130:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476914432:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787829125.322135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829125.322146:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476914688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829125.322155:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829125.322158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829125.322159:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829125.322162:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3492 00000100:00080000:1.0:1787829125.322165:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829125.322168:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829125.322172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829125.322182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476914816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829125.322188:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829125.322191:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829125.322193:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829125.322196:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3492 00000100:00080000:1.0:1787829125.322199:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829125.322202:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829125.322205:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829125.322213:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829125.322220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476914944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829125.322289:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff9a7890b1e580 x1874671476914432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829125.322311:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476914432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829125.322324:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829125.322329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829125.322333:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476914432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829125.322363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829125.322392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829125.322420:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff9a7890b1d680 x1874671476914560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829125.322434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476914560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829125.322444:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829125.322448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829125.322452:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829125.322460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829125.322485:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829125.322513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff9a7890b1de00 x1874671476914688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829125.322526:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476914688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829125.322533:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829125.322535:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829125.322538:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829125.322544:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829125.322568:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829125.322581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff9a7890b1e940 x1874671476914816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829125.322594:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476914816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829125.322605:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829125.322607:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829125.322610:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829125.322617:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829125.322640:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829125.322653:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829125/real 1787829125] req@ffff9a7890b1da40 x1874671476914944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829125.322665:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476914944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829125.322675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829125.322678:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829125.322682:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476914944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829126.346154:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829127.370050:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829128.393961:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829129.418037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829130.441944:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829130.441955:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829130.441960:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829130.441972:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3497 00000100:00080000:2.0:1787829130.441988:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829130.441995:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829130.441997:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829130.442072:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476915072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829130.442103:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829130.442106:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829130.442108:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829130.442111:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3497 00000100:00080000:2.0:1787829130.442115:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829130.442119:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829130.442130:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829130.442141:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476915200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829130.442150:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829130.442154:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829130.442155:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476915072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829130.442156:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829130.442160:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3497 00000100:00080000:2.0:1787829130.442163:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829130.442167:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829130.442173:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829130.442185:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476915328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829130.442195:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829130.442200:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829130.442202:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829130.442206:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3497 00000100:00080000:2.0:1787829130.442210:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829130.442215:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829130.442221:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829130.442233:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476915456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829130.442242:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829130.442245:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829130.442248:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829130.442251:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829130.442255:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3497 00000100:00080000:2.0:1787829130.442260:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829130.442263:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829130.442267:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829130.442276:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476915584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829130.442325:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9a78841d9e00 x1874671476915072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.442342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476915072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.442350:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.442354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829130.442357:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476915072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829130.442395:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829130.442416:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829130.442427:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9a78841da580 x1874671476915200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.442435:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476915200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.442441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.442444:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829130.442447:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829130.442451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829130.442463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829130.442469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9a78841d8000 x1874671476915328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.442477:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476915328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.442482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.442484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829130.442486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829130.442489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829130.442503:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829130.442509:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9a78841d83c0 x1874671476915456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.442516:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476915456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.442524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.442527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829130.442529:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829130.442532:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829130.442542:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829130.442547:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9a78841db480 x1874671476915584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829130.442564:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476915584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829130.442574:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829130.442576:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829130.442578:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829131.466014:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829132.490030:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829133.513980:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829134.537989:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829135.562018:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829135.562027:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829135.562031:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829135.562041:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3503 00010000:00010000:2.0:1787829135.562049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829135.562051:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829135.562059:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829135.562099:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476915712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829135.562126:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829135.562129:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829135.562130:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829135.562133:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3503 00000100:00080000:1.0:1787829135.562135:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829135.562137:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829135.562145:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829135.562153:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476915840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829135.562160:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829135.562162:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829135.562164:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829135.562166:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3503 00000100:00080000:1.0:1787829135.562169:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829135.562171:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829135.562175:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829135.562186:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476915968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829135.562186:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476915712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829135.562210:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829135.562213:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829135.562214:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829135.562218:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3503 00000100:00080000:1.0:1787829135.562220:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829135.562222:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829135.562225:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829135.562232:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476916096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829135.562237:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829135.562240:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829135.562242:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829135.562249:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3503 00000100:00080000:1.0:1787829135.562273:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829135.562276:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829135.562280:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787829135.562284:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829135.562289:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476916224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829135.562355:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9a7890b1da40 x1874671476915712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829135.562374:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476915712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829135.562383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829135.562387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829135.562390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476915712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829135.562398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829135.562417:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829135.562430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9a7890b1e940 x1874671476915840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829135.562443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476915840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829135.562453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829135.562457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829135.562461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829135.562492:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829135.562515:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829135.562528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9a7890b1de00 x1874671476915968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829135.562542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476915968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829135.562548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829135.562551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829135.562554:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476915968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829135.562558:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829135.562574:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829135.562582:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9a7890b1d680 x1874671476916096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829135.562590:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476916096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829135.562597:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829135.562599:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829135.562601:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829135.562605:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829135.562615:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829135.562621:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9a7890b1e580 x1874671476916224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829135.562628:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476916224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829135.562635:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829135.562637:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829135.562638:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829136.585999:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829137.610043:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829138.634022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829139.658003:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829140.681943:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829140.681952:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829140.681956:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829140.681965:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3508 00010000:00010000:0.0:1787829140.681972:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829140.681974:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829140.681982:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829140.682013:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476916352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829140.682036:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829140.682039:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829140.682040:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829140.682043:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3508 00000100:00080000:3.0:1787829140.682045:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829140.682048:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829140.682055:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829140.682064:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476916480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829140.682071:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829140.682073:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829140.682074:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829140.682077:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3508 00000100:00080000:3.0:1787829140.682079:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829140.682081:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829140.682084:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829140.682088:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476916352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829140.682094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476916608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829140.682102:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829140.682104:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829140.682106:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829140.682109:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3508 00000100:00080000:3.0:1787829140.682111:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829140.682113:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829140.682116:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829140.682124:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476916736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829140.682130:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829140.682132:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829140.682133:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829140.682136:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3508 00000100:00080000:3.0:1787829140.682138:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829140.682140:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829140.682143:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829140.682148:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829140.682152:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476916864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829140.682196:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829140/real 1787829140] req@ffff9a78a6776d00 x1874671476916352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829140.682211:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476916352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829140.682219:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829140.682222:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829140.682224:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476916352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829140.682230:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829140.682245:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829140.682253:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829140/real 1787829140] req@ffff9a78a6775680 x1874671476916480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829140.682266:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476916480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829140.682272:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829140.682274:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829140.682276:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829140.682295:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829140.682323:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829140.682333:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829140/real 1787829140] req@ffff9a78a6775e00 x1874671476916608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829140.682358:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476916608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829140.682366:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829140.682370:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829140.682374:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829140.682380:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829140.682411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829140.682422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829140/real 1787829140] req@ffff9a78a6777840 x1874671476916736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829140.682430:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476916736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829140.682441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829140.682443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829140.682446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829140.682450:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829140.682469:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829140.682476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829140/real 1787829140] req@ffff9a78a6774f00 x1874671476916864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829140.682483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476916864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829140.682489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829140.682491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829140.682493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476916864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829141.706001:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829142.729936:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829143.754000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829144.778000:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829145.801890:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829145.801897:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829145.801901:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829145.801908:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3513 00000100:00080000:3.0:1787829145.801917:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829145.801923:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829145.801957:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476916992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829145.801986:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829145.801990:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829145.801992:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829145.801996:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3513 00000100:00080000:3.0:1787829145.801999:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829145.802002:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829145.802011:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829145.802023:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476917120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829145.802028:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476916992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829145.802038:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829145.802041:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829145.802043:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829145.802046:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3513 00000100:00080000:3.0:1787829145.802049:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829145.802051:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829145.802054:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829145.802065:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476917248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829145.802072:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829145.802075:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829145.802076:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829145.802079:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3513 00000100:00080000:3.0:1787829145.802081:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829145.802083:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829145.802087:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829145.802093:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829145.802098:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476917376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829145.802107:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829145.802109:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829145.802111:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829145.802115:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3513 00000100:00080000:3.0:1787829145.802117:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829145.802120:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829145.802124:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829145.802135:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476917504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829145.802151:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829145/real 1787829145] req@ffff9a78a67743c0 x1874671476916992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829145.802175:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476916992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829145.802187:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829145.802193:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829145.802199:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476916992:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829145.802209:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829145.802227:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829145.802255:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829145.802286:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829145/real 1787829145] req@ffff9a78a6777c00 x1874671476917120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829145.802301:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476917120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829145.802310:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829145.802314:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829145.802317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829145.802324:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829145.802346:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829145.802358:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829145/real 1787829145] req@ffff9a78a6774f00 x1874671476917248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829145.802370:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476917248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829145.802380:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829145.802383:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829145.802387:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829145.802393:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829145.802409:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829145.802426:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829145/real 1787829145] req@ffff9a78a6777840 x1874671476917376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829145.802438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476917376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829145.802448:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829145.802451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829145.802455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829145.802461:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829145.802481:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829145.802492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829145/real 1787829145] req@ffff9a78a6775e00 x1874671476917504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829145.802505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476917504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829145.802516:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829145.802519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829145.802523:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829146.826000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829147.849993:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829148.873970:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829149.897924:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829150.921917:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829150.921925:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829150.921928:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829150.921937:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3518 00000100:00080000:1.0:1787829150.921946:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829150.921952:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829150.921986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829150.922002:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476917632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829150.922023:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829150.922026:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829150.922027:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829150.922030:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3518 00000100:00080000:1.0:1787829150.922033:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829150.922036:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829150.922060:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829150.922066:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476917632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829150.922072:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476917760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829150.922080:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829150.922088:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829150.922090:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829150.922093:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3518 00000100:00080000:1.0:1787829150.922097:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829150.922100:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829150.922103:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829150.922128:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476917888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829150.922136:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1787829150.922140:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829150.922141:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829150.922145:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829150.922149:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3518 00000100:00080000:1.0:1787829150.922154:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829150.922157:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829150.922162:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829150.922197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476918016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829150.922204:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829150.922206:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829150.922207:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829150.922210:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3518 00000100:00080000:1.0:1787829150.922213:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829150.922215:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829150.922218:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829150.922224:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829150/real 1787829150] req@ffff9a7890b1f480 x1874671476917632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829150.922237:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476918144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829150.922275:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476917632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829150.922283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829150.922286:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829150.922289:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476917632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829150.922309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829150.922337:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829150.922346:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829150/real 1787829150] req@ffff9a7890b1f840 x1874671476917760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829150.922355:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476917760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829150.922360:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829150.922362:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829150.922364:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829150.922368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829150.922378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829150.922384:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829150/real 1787829150] req@ffff9a7890b1fc00 x1874671476917888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829150.922400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476917888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829150.922409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829150.922411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829150.922414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476917888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829150.922417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829150.922426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829150.922432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829150/real 1787829150] req@ffff9a7890b1c000 x1874671476918016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829150.922439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476918016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829150.922445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829150.922447:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829150.922449:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829150.922452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829150.922461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829150.922466:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829150/real 1787829150] req@ffff9a7890b1f0c0 x1874671476918144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829150.922473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476918144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829150.922479:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829150.922481:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829150.922483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829151.945932:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829152.969933:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829153.993925:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829155.017928:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829156.041914:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.041924:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829156.041928:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.041937:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3523 00000100:00080000:1.0:1787829156.041948:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829156.041955:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829156.041996:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476918272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829156.042006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829156.042029:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.042036:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829156.042038:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.042042:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3523 00000100:00080000:1.0:1787829156.042046:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829156.042050:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829156.042060:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829156.042076:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476918400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829156.042084:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476918272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829156.042096:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.042101:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829156.042103:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.042108:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3523 00000100:00080000:1.0:1787829156.042111:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829156.042114:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829156.042118:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829156.042130:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476918528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829156.042137:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.042140:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829156.042142:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.042144:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3523 00000100:00080000:1.0:1787829156.042147:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787829156.042152:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829156.042154:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829156.042160:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829156.042169:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476918656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829156.042175:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.042178:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829156.042179:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.042182:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3523 00000100:00080000:1.0:1787829156.042184:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829156.042186:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829156.042189:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829156.042212:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476918784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829156.042230:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9a7890b1f0c0 x1874671476918272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.042250:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476918272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.042262:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.042266:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829156.042270:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476918272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829156.042291:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829156.042311:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829156.042323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9a7890b1c000 x1874671476918400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.042335:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476918400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.042344:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.042347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829156.042351:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829156.042357:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829156.042374:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829156.042395:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9a7890b1fc00 x1874671476918528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.042407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476918528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.042416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.042419:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829156.042423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829156.042428:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829156.042444:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829156.042453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9a7890b1f840 x1874671476918656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.042464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476918656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.042474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.042477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829156.042480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829156.042485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829156.042499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829156.042508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9a7890b1f480 x1874671476918784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.042527:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476918784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.042537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.042539:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829156.042543:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476918784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829157.066075:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829158.090019:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829159.114068:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829160.137997:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829161.161933:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829161.161943:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829161.161948:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829161.161958:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3528 00000100:00080000:1.0:1787829161.161966:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829161.161971:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829161.162002:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476918912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829161.162005:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829161.162025:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829161.162029:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829161.162030:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829161.162033:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3528 00000100:00080000:1.0:1787829161.162036:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829161.162039:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829161.162047:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829161.162067:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476919040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829161.162075:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829161.162077:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829161.162079:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829161.162086:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3528 00000100:00080000:1.0:1787829161.162088:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829161.162090:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829161.162093:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829161.162099:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476919168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829161.162108:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829161.162110:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829161.162111:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829161.162114:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3528 00000100:00080000:1.0:1787829161.162116:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829161.162123:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829161.162127:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476918912:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787829161.162128:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829161.162149:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476919296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829161.162190:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829161.162192:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829161.162194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829161.162198:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3528 00000100:00080000:1.0:1787829161.162200:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829161.162203:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829161.162207:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829161.162217:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476919424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829161.162270:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829161.162325:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9a7890b1e580 x1874671476918912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.162341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476918912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.162351:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.162354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829161.162357:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476918912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829161.162364:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829161.162380:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829161.162388:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9a7890b1d680 x1874671476919040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.162396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476919040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.162402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.162405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829161.162408:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829161.162432:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829161.162442:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829161.162448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9a7890b1de00 x1874671476919168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.162474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476919168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.162480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.162482:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829161.162484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829161.162489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829161.162513:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829161.162521:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9a7890b1e940 x1874671476919296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.162529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476919296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.162537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.162540:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829161.162543:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829161.162548:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829161.162560:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829161.162566:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9a7890b1da40 x1874671476919424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.162574:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476919424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.162582:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.162584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829161.162586:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829162.186104:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829163.210083:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829164.234023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829165.257973:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829166.281976:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787829166.281982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829166.281995:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829166.282002:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829166.282015:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3533 00000100:00080000:2.0:1787829166.282029:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829166.282037:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829166.282084:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476919552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829166.282118:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829166.282133:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829166.282136:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829166.282141:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3533 00000100:00080000:2.0:1787829166.282146:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829166.282151:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829166.282163:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829166.282172:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476919552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829166.282182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476919680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829166.282197:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829166.282202:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829166.282204:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829166.282210:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3533 00000100:00080000:2.0:1787829166.282215:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829166.282221:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829166.282226:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829166.282239:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476919808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829166.282252:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787829166.282258:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829166.282260:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829166.282264:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829166.282271:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3533 00000100:00080000:2.0:1787829166.282277:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829166.282283:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829166.282289:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829166.282302:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476919936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829166.282321:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829166.282324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9a78841db840 x1874671476919552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829166.282327:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829166.282335:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829166.282349:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3533 00000100:00080000:2.0:1787829166.282352:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829166.282356:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476919552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829166.282361:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829166.282375:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829166.282378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829166.282384:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829166.282388:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476919552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829166.282390:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476920064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829166.282423:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829166.282445:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829166.282454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9a78841d92c0 x1874671476919680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829166.282462:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476919680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829166.282467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829166.282470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829166.282472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829166.282476:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829166.282486:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829166.282492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9a78841d9680 x1874671476919808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829166.282499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476919808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829166.282504:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829166.282506:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829166.282508:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829166.282512:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829166.282532:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829166.282539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9a78841d8b40 x1874671476919936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829166.282551:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476919936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829166.282557:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829166.282559:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829166.282561:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476919936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829166.282565:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829166.282576:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829166.282582:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9a78841da940 x1874671476920064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829166.282589:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476920064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829166.282595:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829166.282597:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829166.282599:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829167.306010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829168.330094:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829169.354040:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829170.377976:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829171.402061:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829171.402076:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829171.402083:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.402122:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3538 00000100:00080000:3.0:1787829171.402155:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829171.402165:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829171.402224:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476920192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829171.402259:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829171.402263:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829171.402265:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.402269:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3538 00000100:00080000:3.0:1787829171.402272:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829171.402276:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829171.402288:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829171.402301:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476920320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829171.402311:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829171.402315:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829171.402318:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.402327:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3538 00000100:00100000:2.0:1787829171.402330:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476920192:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829171.402331:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829171.402339:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829171.402350:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829171.402362:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476920448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829171.402371:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829171.402374:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829171.402376:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.402380:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3538 00000100:00080000:3.0:1787829171.402384:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829171.402388:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829171.402393:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829171.402402:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476920576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829171.402410:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829171.402413:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829171.402415:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829171.402419:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3538 00000100:00080000:3.0:1787829171.402423:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787829171.402430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787829171.402438:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829171.402451:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829171.402488:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476920704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829171.402500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9a78a6775680 x1874671476920192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829171.402523:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476920192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829171.402534:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829171.402539:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829171.402544:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476920192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829171.402605:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829171.402634:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:0.0:1787829171.402657:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829171.402659:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9a78a6776d00 x1874671476920320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829171.402672:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476920320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829171.402680:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829171.402684:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829171.402687:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829171.402693:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829171.402706:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829171.402714:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9a78a67761c0 x1874671476920448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829171.402724:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476920448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829171.402731:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829171.402734:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829171.402737:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829171.402742:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829171.402754:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829171.402762:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9a78a6774780 x1874671476920576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829171.402770:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476920576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829171.402783:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829171.402786:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829171.402788:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829171.402793:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829171.402804:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829171.402811:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9a78a67770c0 x1874671476920704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829171.402819:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476920704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829171.402826:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829171.402850:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829171.402852:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829172.426099:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829173.449954:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829174.473980:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829175.498024:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829176.521918:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829176.521926:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829176.521930:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829176.521938:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3544 00000100:00080000:3.0:1787829176.521946:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829176.521953:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829176.521984:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476920832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829176.522006:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829176.522011:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829176.522012:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829176.522015:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3544 00000100:00080000:3.0:1787829176.522017:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829176.522019:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829176.522027:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829176.522036:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476920960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829176.522043:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829176.522046:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829176.522047:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829176.522078:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476920832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829176.522088:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3544 00000100:00080000:3.0:1787829176.522093:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829176.522098:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829176.522104:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829176.522122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476921088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829176.522143:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1787829176.522148:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787829176.522151:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829176.522155:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829176.522161:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3544 00000100:00080000:3.0:1787829176.522167:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829176.522172:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829176.522179:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829176.522192:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476921216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829176.522202:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829176.522206:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829176.522208:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829176.522211:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829176/real 1787829176] req@ffff9a78a6776940 x1874671476920832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829176.522216:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3544 00000100:00080000:3.0:1787829176.522223:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829176.522232:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829176.522239:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829176.522244:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476920832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829176.522256:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476921344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829176.522257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829176.522267:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829176.522277:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476920832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829176.522288:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829176.522318:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829176.522337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829176/real 1787829176] req@ffff9a78a6776580 x1874671476920960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787829176.522345:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829176.522355:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476920960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829176.522362:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829176.522365:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829176.522367:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476920960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829176.522386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829176.522402:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829176.522413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829176/real 1787829176] req@ffff9a78a6774b40 x1874671476921088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829176.522421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476921088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829176.522426:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829176.522429:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829176.522431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829176.522435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829176.522447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829176.522454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829176/real 1787829176] req@ffff9a78a67752c0 x1874671476921216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829176.522461:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476921216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829176.522467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829176.522469:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829176.522470:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829176.522474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829176.522483:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829176.522489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829176/real 1787829176] req@ffff9a78a6775a40 x1874671476921344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829176.522504:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476921344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829176.522511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829176.522513:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829176.522515:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829177.546092:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829178.570018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829179.593957:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829180.618014:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829181.641907:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829181.641917:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829181.641921:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829181.641930:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3549 00000100:00080000:3.0:1787829181.641938:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829181.641943:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829181.641974:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476921472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829181.641996:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829181.641999:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829181.642000:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829181.642003:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3549 00000100:00080000:3.0:1787829181.642006:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829181.642008:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829181.642031:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829181.642042:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476921600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829181.642052:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476921472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829181.642052:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829181.642061:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829181.642063:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829181.642067:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3549 00000100:00080000:3.0:1787829181.642070:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829181.642072:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829181.642076:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829181.642086:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476921728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829181.642093:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829181.642095:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829181.642096:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829181.642098:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3549 00000100:00080000:3.0:1787829181.642103:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829181.642106:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829181.642112:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829181.642113:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829181.642125:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476921856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829181.642136:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829181.642139:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829181.642140:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829181.642143:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3549 00000100:00080000:3.0:1787829181.642146:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829181.642149:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829181.642153:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829181.642157:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829181/real 1787829181] req@ffff9a78a6777480 x1874671476921472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829181.642162:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476921984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829181.642190:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476921472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829181.642200:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829181.642204:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829181.642207:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476921472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829181.642227:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829181.642246:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:1.0:1787829181.642254:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829181.642260:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829181/real 1787829181] req@ffff9a78a6774000 x1874671476921600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829181.642276:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476921600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829181.642285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829181.642288:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829181.642292:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829181.642298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829181.642324:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829181.642337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829181/real 1787829181] req@ffff9a78a6775a40 x1874671476921728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829181.642352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476921728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829181.642361:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829181.642364:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829181.642369:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829181.642375:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829181.642397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829181.642410:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829181/real 1787829181] req@ffff9a78a67752c0 x1874671476921856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829181.642423:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476921856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829181.642434:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829181.642438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829181.642442:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829181.642448:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829181.642470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829181.642481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829181/real 1787829181] req@ffff9a78a6774b40 x1874671476921984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829181.642495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476921984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829181.642505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829181.642509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829181.642512:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476921984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829182.665984:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829183.689982:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829184.713983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829185.738093:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829186.761947:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829186.761954:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829186.761958:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829186.761968:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3554 00000100:00080000:1.0:1787829186.761992:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829186.762001:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829186.762003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829186.762060:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476922112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829186.762089:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829186.762093:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829186.762094:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829186.762098:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3554 00000100:00080000:1.0:1787829186.762102:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829186.762106:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829186.762118:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829186.762143:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476922240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829186.762143:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476922112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829186.762172:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829186.762177:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829186.762181:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829186.762188:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3554 00000100:00080000:1.0:1787829186.762194:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829186.762200:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829186.762208:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829186.762225:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476922368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829186.762236:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829186.762239:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829186.762241:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829186.762246:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3554 00000100:00080000:1.0:1787829186.762250:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829186.762254:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829186.762262:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829186.762275:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476922496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829186.762288:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829186.762291:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829186.762293:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829186.762296:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3554 00000100:00080000:1.0:1787829186.762297:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829186.762300:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829186.762309:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829186.762313:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829186.762341:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476922624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829186.762372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829186.762389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829186.762417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829187.786026:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829188.810029:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829189.834159:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829189.834180:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787829189.834235:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829189.834253:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829189.834293:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829189.834308:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829189.834324:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829189.834332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829186/real 1787829189] req@ffff9a7890b1e940 x1874671476922240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000100:2.0:1787829189.834340:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829189.834371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476922240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829189.834386:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829189.834393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829189.834399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829189.834408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829186/real 1787829189] req@ffff9a7890b1de00 x1874671476922368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829189.834434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476922368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829189.834449:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829189.834452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829189.834456:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829189.834463:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829186/real 1787829189] req@ffff9a7890b1d680 x1874671476922496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829189.834475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476922496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829189.834487:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829189.834491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829189.834495:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829189.834502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829186/real 1787829189] req@ffff9a7890b1da40 x1874671476922112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829189.834516:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476922112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829189.834523:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829189.834528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829189.834531:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476922112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829189.834580:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829186/real 1787829189] req@ffff9a7890b1e580 x1874671476922624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829189.834595:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476922624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829189.834610:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829189.834615:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829189.834618:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829190.858023:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829191.881942:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829191.881974:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829191.881980:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829191.881984:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829191.881994:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3559 00000100:00080000:2.0:1787829191.882004:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829191.882010:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829191.882039:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476922752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829191.882061:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829191.882064:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829191.882065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829191.882068:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3559 00000100:00080000:2.0:1787829191.882071:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829191.882075:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829191.882084:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829191.882095:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476922752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829191.882096:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476922880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829191.882110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829191.882113:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829191.882116:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829191.882121:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3559 00000100:00080000:2.0:1787829191.882126:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829191.882129:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829191.882132:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829191.882142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476923008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829191.882150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829191.882153:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829191.882157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829191.882163:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829191.882169:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3559 00000100:00080000:2.0:1787829191.882175:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829191.882180:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829191.882190:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829191.882203:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476923136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829191.882213:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829191.882218:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829191.882220:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829191.882225:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829191/real 1787829191] req@ffff9a78841da940 x1874671476922752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829191.882227:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3559 00000100:00080000:2.0:1787829191.882234:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829191.882242:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829191.882250:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829191.882258:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476922752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829191.882265:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476923264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829191.882285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829191.882292:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829191.882297:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476922752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829191.882323:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829191.882349:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829191.882362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829191/real 1787829191] req@ffff9a78841d8b40 x1874671476922880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829191.882376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476922880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829191.882383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829191.882387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829191.882391:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476922880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829191.882398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829191.882419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829191.882431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829191/real 1787829191] req@ffff9a78841d9680 x1874671476923008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829191.882443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476923008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829191.882451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829191.882454:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829191.882458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829191.882464:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829191.882484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829191.882496:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829191/real 1787829191] req@ffff9a78841d92c0 x1874671476923136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829191.882508:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476923136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829191.882520:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829191.882523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829191.882527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829191.882534:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829191.882561:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829191.882573:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829191/real 1787829191] req@ffff9a78841db840 x1874671476923264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829191.882585:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476923264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829191.882596:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829191.882599:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829191.882603:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829192.906131:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829193.930128:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829194.953973:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829195.978029:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829197.001980:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787829197.001990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829197.001991:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829197.002015:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829197.002028:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3564 00000100:00080000:1.0:1787829197.002040:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829197.002047:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829197.002098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476923392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829197.002127:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829197.002132:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829197.002134:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829197.002138:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3564 00000100:00080000:1.0:1787829197.002143:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829197.002148:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829197.002159:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829197.002177:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476923520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829197.002179:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476923392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829197.002215:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829197.002218:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829197.002219:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829197.002222:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3564 00000100:00080000:1.0:1787829197.002225:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829197.002227:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829197.002231:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829197.002240:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476923648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829197.002246:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829197.002248:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829197.002249:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829197.002253:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3564 00000100:00080000:1.0:1787829197.002260:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829197.002269:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829197.002273:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829197.002276:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829197.002286:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476923776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829197.002293:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829197.002296:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829197.002297:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829197.002299:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3564 00000100:00080000:1.0:1787829197.002302:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829197.002305:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829197.002308:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829197.002319:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff9a7890b1f480 x1874671476923392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829197.002320:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476923904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829197.002364:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476923392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829197.002375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829197.002380:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829197.002384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476923392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829197.002407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829197.002426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829197.002436:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff9a7890b1f840 x1874671476923520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829197.002444:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476923520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829197.002451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829197.002453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829197.002456:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829197.002460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829197.002470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829197.002476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff9a7890b1fc00 x1874671476923648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829197.002484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476923648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829197.002489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829197.002491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829197.002493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829197.002496:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829197.002507:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829197.002512:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff9a7890b1c000 x1874671476923776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829197.002520:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476923776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829197.002525:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829197.002527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829197.002530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829197.002533:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829197.002543:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829197.002548:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829197/real 1787829197] req@ffff9a7890b1f0c0 x1874671476923904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829197.002555:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476923904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829197.002561:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829197.002563:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829197.002565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476923904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829198.026094:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829199.050018:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829200.073965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829201.098060:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829202.121947:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.121956:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829202.121961:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.121969:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3569 00000100:00080000:1.0:1787829202.121979:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.121984:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829202.122019:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829202.122024:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476924032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829202.122053:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.122057:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829202.122059:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.122062:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3569 00000100:00080000:1.0:1787829202.122067:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.122071:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829202.122090:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829202.122102:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476924160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829202.122108:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476924032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829202.122112:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.122117:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829202.122119:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.122123:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3569 00000100:00080000:1.0:1787829202.122125:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.122127:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829202.122131:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829202.122140:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476924288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829202.122147:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.122149:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829202.122151:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.122153:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3569 00000100:00080000:1.0:1787829202.122163:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.122165:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829202.122168:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829202.122187:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829202.122190:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476924416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829202.122203:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829202.122206:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829202.122207:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829202.122210:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3569 00000100:00080000:1.0:1787829202.122212:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829202.122214:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829202.122217:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829202.122232:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476924544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829202.122242:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9a7890b1f0c0 x1874671476924032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829202.122262:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476924032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829202.122274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829202.122286:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829202.122291:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476924032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829202.122314:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829202.122338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829202.122351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9a7890b1c000 x1874671476924160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829202.122364:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476924160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829202.122373:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829202.122377:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829202.122380:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829202.122387:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829202.122405:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829202.122415:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9a7890b1fc00 x1874671476924288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829202.122428:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476924288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829202.122436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829202.122440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829202.122443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829202.122450:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829202.122468:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829202.122478:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9a7890b1f840 x1874671476924416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829202.122498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476924416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829202.122509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829202.122512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829202.122516:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829202.122522:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829202.122542:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829202.122554:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829202/real 1787829202] req@ffff9a7890b1f480 x1874671476924544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829202.122567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476924544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829202.122577:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829202.122581:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829202.122584:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829203.146003:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829204.169989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829205.193993:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829206.218021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829207.241954:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.241962:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829207.241966:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.241992:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3574 00010000:00010000:3.0:1787829207.242004:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829207.242005:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.242017:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829207.242059:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476924672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829207.242085:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.242089:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829207.242090:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.242094:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3574 00000100:00080000:1.0:1787829207.242098:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.242102:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829207.242112:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829207.242131:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476924672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829207.242132:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476924800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829207.242167:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.242171:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829207.242173:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.242177:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3574 00000100:00080000:1.0:1787829207.242182:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.242186:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829207.242190:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829207.242200:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476924928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829207.242208:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.242210:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829207.242211:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.242216:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3574 00000800:00000100:3.0:1787829207.242218:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829207.242220:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.242224:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829207.242228:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829207.242237:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476925056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829207.242249:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829207.242252:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829207.242255:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829207.242259:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3574 00000100:00080000:1.0:1787829207.242264:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829207.242268:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829207.242273:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829207.242281:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829207] req@ffff9a7890b1e580 x1874671476924672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829207.242289:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476925184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829207.242300:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476924672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829207.242321:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829207.242324:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829207.242328:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476924672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829207.242348:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829207.242363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829207.242371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829207] req@ffff9a7890b1da40 x1874671476924800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829207.242379:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476924800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829207.242384:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829207.242386:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829207.242388:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829207.242399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829207.242409:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829207.242414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829207] req@ffff9a7890b1d680 x1874671476924928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829207.242439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476924928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829207.242445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829207.242447:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829207.242450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476924928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829207.242465:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829207.242478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829207.242501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829207] req@ffff9a7890b1de00 x1874671476925056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829207.242508:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476925056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829207.242514:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829207.242516:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829207.242517:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829207.242530:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829207.242539:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829207.242545:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829207/real 1787829207] req@ffff9a7890b1e940 x1874671476925184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829207.242552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476925184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829207.242558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829207.242560:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829207.242562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829208.265973:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829209.289986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829210.313911:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829211.338004:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829212.361966:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829212.361975:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:2.0:1787829212.361979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829212.361980:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829212.361990:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3579 00000100:00080000:3.0:1787829212.362000:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829212.362006:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829212.362037:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476925312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829212.362065:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829212.362069:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829212.362071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829212.362076:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3579 00000100:00080000:3.0:1787829212.362081:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829212.362086:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829212.362096:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829212.362105:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476925312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829212.362121:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476925440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829212.362134:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829212.362139:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829212.362141:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829212.362146:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3579 00000100:00080000:3.0:1787829212.362160:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829212.362164:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829212.362170:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829212.362172:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829212.362187:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476925568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829212.362203:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829212.362206:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829212.362207:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829212.362211:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3579 00000100:00080000:3.0:1787829212.362214:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829212.362217:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829212.362222:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829212.362229:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9a78a6776580 x1874671476925312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829212.362233:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476925696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829212.362258:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829212.362260:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829212.362261:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829212.362264:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3579 00000100:00080000:3.0:1787829212.362267:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829212.362270:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829212.362273:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829212.362276:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476925312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829212.362284:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476925824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829212.362305:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829212.362311:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829212.362315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476925312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829212.362343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829212.362363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829212.362373:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9a78a6776940 x1874671476925440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829212.362382:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476925440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829212.362388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829212.362391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829212.362393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829212.362397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829212.362408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829212.362415:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9a78a67770c0 x1874671476925568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829212.362422:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476925568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829212.362428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829212.362429:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829212.362431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829212.362435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829212.362445:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829212.362451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9a78a6774780 x1874671476925696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829212.362463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476925696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829212.362470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829212.362471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829212.362473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829212.362477:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829212.362486:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829212.362492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829212/real 1787829212] req@ffff9a78a67761c0 x1874671476925824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829212.362499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476925824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829212.362505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829212.362507:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829212.362509:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476925824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829213.385946:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829214.409943:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829215.433962:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829216.457928:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829217.481911:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829217.481921:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829217.481925:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787829217.481931:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829217.481937:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3584 00000100:00080000:1.0:1787829217.481956:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829217.481963:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829217.481995:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476925952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829217.482015:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829217.482018:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829217.482022:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829217.482025:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3584 00000100:00080000:1.0:1787829217.482028:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829217.482031:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829217.482038:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829217.482047:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476926080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829217.482060:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829217.482063:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829217.482063:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476925952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829217.482068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829217.482075:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3584 00000100:00080000:1.0:1787829217.482077:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829217.482080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829217.482083:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829217.482093:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476926208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829217.482100:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829217.482103:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829217.482105:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829217.482107:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3584 00000100:00080000:1.0:1787829217.482116:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829217.482118:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829217.482122:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829217.482126:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829217.482133:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476926336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829217.482142:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829217.482144:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829217.482145:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829217.482148:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3584 00000100:00080000:1.0:1787829217.482152:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829217.482155:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829217.482158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829217.482167:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829217/real 1787829217] req@ffff9a7890b1e940 x1874671476925952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829217.482169:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476926464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829217.482211:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476925952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829217.482221:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829217.482225:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829217.482228:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476925952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829217.482247:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829217.482263:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829217.482271:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829217/real 1787829217] req@ffff9a7890b1de00 x1874671476926080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829217.482280:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476926080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829217.482285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829217.482287:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829217.482289:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829217.482294:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829217.482304:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829217.482309:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829217/real 1787829217] req@ffff9a7890b1d680 x1874671476926208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829217.482320:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476926208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829217.482328:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829217.482332:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829217.482336:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829217.482342:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829217.482363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829217.482376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829217/real 1787829217] req@ffff9a7890b1da40 x1874671476926336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829217.482385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476926336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829217.482391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829217.482393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829217.482397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829217.482403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829217.482423:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829217.482435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829217/real 1787829217] req@ffff9a7890b1e580 x1874671476926464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829217.482448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476926464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829217.482456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829217.482458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829217.482460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829218.505946:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829219.529892:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829220.553940:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829221.577931:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829222.601938:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829222.601946:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829222.601950:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829222.601959:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3590 00010000:00010000:3.0:1787829222.601966:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829222.601968:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829222.601975:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829222.602007:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476926592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829222.602027:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829222.602030:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829222.602031:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829222.602034:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3590 00000100:00080000:1.0:1787829222.602036:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829222.602046:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829222.602054:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829222.602063:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476926720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829222.602072:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476926592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829222.602073:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829222.602081:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829222.602083:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829222.602086:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3590 00000100:00080000:1.0:1787829222.602089:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829222.602091:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829222.602094:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829222.602103:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476926848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829222.602110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829222.602113:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829222.602114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829222.602118:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3590 00000100:00080000:1.0:1787829222.602130:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829222.602137:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787829222.602138:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829222.602146:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829222.602164:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476926976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829222.602173:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829222.602177:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829222.602179:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829222.602184:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3590 00000100:00100000:2.0:1787829222.602188:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829222/real 1787829222] req@ffff9a7890b1f480 x1874671476926592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829222.602190:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829222.602200:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829222.602210:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829222.602219:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476926592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829222.602225:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476927104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829222.602240:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829222.602248:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829222.602254:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476926592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829222.602279:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829222.602298:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829222.602308:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829222/real 1787829222] req@ffff9a7890b1f840 x1874671476926720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829222.602317:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476926720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829222.602323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829222.602326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829222.602328:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829222.602333:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829222.602343:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829222.602349:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829222/real 1787829222] req@ffff9a7890b1fc00 x1874671476926848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829222.602369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476926848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829222.602375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829222.602377:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829222.602380:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829222.602384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829222.602395:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829222.602402:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829222/real 1787829222] req@ffff9a7890b1c000 x1874671476926976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829222.602409:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476926976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829222.602416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829222.602417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829222.602419:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476926976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829222.602423:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829222.602432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829222.602438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829222/real 1787829222] req@ffff9a7890b1f0c0 x1874671476927104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829222.602444:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476927104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829222.602455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829222.602457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829222.602460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829223.626027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829224.650003:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829225.673960:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829226.698002:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829227.721929:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829227.721941:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829227.721945:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829227.721955:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3595 00000100:00080000:1.0:1787829227.721965:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829227.721974:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829227.722008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829227.722016:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476927232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829227.722041:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829227.722055:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829227.722057:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829227.722062:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3595 00000100:00080000:1.0:1787829227.722066:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829227.722071:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829227.722079:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476927232:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787829227.722080:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829227.722098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476927360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829227.722111:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829227.722114:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829227.722116:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829227.722120:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3595 00000100:00080000:1.0:1787829227.722125:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829227.722129:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787829227.722133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829227.722137:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829227.722153:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476927488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829227.722174:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829227.722178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829227.722180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829227.722187:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3595 00000100:00100000:2.0:1787829227.722187:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829227/real 1787829227] req@ffff9a7890b1f0c0 x1874671476927232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829227.722195:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829227.722204:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829227.722208:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476927232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829227.722211:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:2.0:1787829227.722219:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829227.722223:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829227.722228:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476927232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829227.722240:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476927616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829227.722248:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927360:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829227.722259:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829227.722264:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829227.722266:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1787829227.722271:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829227.722275:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3595 00000100:00100000:2.0:1787829227.722283:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829227/real 1787829227] req@ffff9a7890b1c000 x1874671476927360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829227.722287:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829227.722296:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829227.722299:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476927360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829227.722303:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787829227.722308:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829227.722312:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829227.722315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829227.722321:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476927744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829227.722322:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829227.722340:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829227.722350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829227/real 1787829227] req@ffff9a7890b1fc00 x1874671476927488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829227.722359:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476927488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829227.722365:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829227.722368:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829227.722370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829227.722380:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829227.722391:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829227.722397:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829227/real 1787829227] req@ffff9a7890b1f840 x1874671476927616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829227.722404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476927616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829227.722410:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829227.722412:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829227.722414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829227.722417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829227.722426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829227.722432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829227/real 1787829227] req@ffff9a7890b1f480 x1874671476927744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829227.722447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476927744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829227.722454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829227.722456:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829227.722458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476927744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829228.745935:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829229.769960:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829230.794029:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829231.817955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829232.842007:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829232.842018:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829232.842023:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829232.842036:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3600 00000100:00080000:2.0:1787829232.842050:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829232.842060:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829232.842062:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829232.842101:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476927872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829232.842135:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829232.842137:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829232.842139:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829232.842142:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3600 00000100:00080000:2.0:1787829232.842145:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829232.842148:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829232.842156:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829232.842168:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476928000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829232.842178:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829232.842181:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829232.842182:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829232.842186:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3600 00000100:00080000:2.0:1787829232.842189:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829232.842196:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829232.842197:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476927872:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829232.842200:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829232.842217:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476928128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829232.842229:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829232.842233:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829232.842235:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829232.842240:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3600 00000100:00080000:2.0:1787829232.842244:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829232.842249:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829232.842256:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829232.842269:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476928256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829232.842277:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829232.842279:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829232.842281:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829232.842284:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3600 00000100:00080000:2.0:1787829232.842294:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829232.842300:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829232.842303:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829232.842307:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829232.842316:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476928384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829232.842382:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829232/real 1787829232] req@ffff9a78841db480 x1874671476927872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829232.842407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476927872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829232.842421:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829232.842427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829232.842432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476927872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829232.842462:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829232.842490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829232.842506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829232/real 1787829232] req@ffff9a78841d83c0 x1874671476928000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829232.842521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476928000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829232.842531:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829232.842535:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829232.842539:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829232.842546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829232.842568:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829232.842581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829232/real 1787829232] req@ffff9a78841d8000 x1874671476928128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829232.842594:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476928128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829232.842604:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829232.842608:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829232.842612:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829232.842619:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829232.842642:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829232.842656:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829232/real 1787829232] req@ffff9a78841da580 x1874671476928256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829232.842686:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476928256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829232.842697:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829232.842700:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829232.842704:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829232.842711:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829232.842733:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829232.842746:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829232/real 1787829232] req@ffff9a78841d9e00 x1874671476928384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829232.842759:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476928384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829232.842770:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829232.842774:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829232.842778:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829233.866004:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829234.889998:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829235.913950:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829236.938000:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829237.961876:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829237.961892:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829237.961898:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829237.961906:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3605 00000100:00080000:2.0:1787829237.961915:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829237.961921:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787829237.961930:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829237.961953:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476928512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829237.961972:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829237.961974:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829237.961976:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829237.961978:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3605 00000100:00080000:2.0:1787829237.961981:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829237.961983:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829237.961991:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829237.962001:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476928640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829237.962010:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829237.962012:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476928512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829237.962018:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829237.962022:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829237.962025:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3605 00000100:00080000:2.0:1787829237.962027:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829237.962029:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829237.962032:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829237.962040:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476928768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829237.962047:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829237.962049:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829237.962051:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829237.962053:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3605 00000100:00080000:2.0:1787829237.962072:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829237.962074:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829237.962077:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829237.962085:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829237.962086:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476928896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829237.962095:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829237.962098:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829237.962099:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829237.962101:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3605 00000100:00080000:2.0:1787829237.962104:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829237.962106:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829237.962110:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829237.962117:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476929024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829237.962155:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829237/real 1787829237] req@ffff9a78841d8780 x1874671476928512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829237.962176:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476928512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829237.962188:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829237.962193:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829237.962197:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476928512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829237.962220:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829237.962243:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829237.962256:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829237/real 1787829237] req@ffff9a78841d9a40 x1874671476928640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829237.962270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476928640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829237.962279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829237.962283:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829237.962287:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829237.962293:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829237.962314:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829237.962326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829237/real 1787829237] req@ffff9a78841da1c0 x1874671476928768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829237.962339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476928768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829237.962347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829237.962351:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829237.962354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829237.962361:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829237.962380:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829237.962391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829237/real 1787829237] req@ffff9a78841db0c0 x1874671476928896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829237.962403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476928896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829237.962413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829237.962417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829237.962420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476928896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829237.962427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829237.962447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829237.962458:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829237/real 1787829237] req@ffff9a78841dbc00 x1874671476929024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829237.962470:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476929024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829262 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829237.962480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829237.962484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829237.962487:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829238.986012:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829240.009980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829241.033983:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829242.058025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829243.081923:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829243.081942:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829243.081946:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829243.081954:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3610 00000100:00080000:1.0:1787829243.081963:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829243.081968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829243.081970:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829243.082014:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476929152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829243.082041:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829243.082045:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829243.082046:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829243.082049:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3610 00000100:00080000:1.0:1787829243.082052:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829243.082055:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829243.082062:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829243.082073:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476929280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829243.082075:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476929152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829243.082087:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829243.082090:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829243.082091:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829243.082094:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3610 00000100:00080000:1.0:1787829243.082097:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829243.082099:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829243.082103:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829243.082112:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476929408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829243.082128:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829243.082131:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829243.082133:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829243.082138:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3610 00000100:00080000:1.0:1787829243.082142:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829243.082149:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829243.082149:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829243.082158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829243.082173:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476929536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829243.082183:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829243.082186:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829243.082188:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829243.082191:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3610 00000100:00080000:1.0:1787829243.082194:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829243.082199:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829243.082199:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9a7890b1e580 x1874671476929152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829243.082206:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829243.082220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476929664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829243.082224:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476929152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829243.082243:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829243.082247:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829243.082251:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476929152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829243.082269:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829243.082285:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829243.082293:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9a7890b1da40 x1874671476929280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829243.082306:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476929280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829243.082312:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829243.082314:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829243.082316:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829243.082320:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829243.082329:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829243.082335:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9a7890b1d680 x1874671476929408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829243.082342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476929408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829243.082347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829243.082349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829243.082351:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829243.082354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829243.082372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829243.082378:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9a7890b1de00 x1874671476929536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829243.082385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476929536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829243.082395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829243.082397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829243.082399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829243.082402:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829243.082411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829243.082419:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9a7890b1e940 x1874671476929664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829243.082442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476929664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829243.082452:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829243.082455:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829243.082459:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829244.106058:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829245.130064:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829246.153979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829247.178130:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829248.202033:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829248.202054:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829248.202066:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829248.202088:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3615 00000100:00080000:1.0:1787829248.202107:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829248.202129:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829248.202142:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829248.202210:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476929792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829248.202242:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829248.202257:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829248.202259:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829248.202263:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3615 00000100:00080000:1.0:1787829248.202267:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829248.202274:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829248.202305:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829248.202330:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476929792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829248.202341:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476929920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829248.202371:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829248.202411:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829248.202414:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829248.202422:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3615 00000100:00080000:1.0:1787829248.202439:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829248.202445:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829248.202470:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829248.202472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829248.202495:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476930048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829248.202522:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829248.202526:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829248.202529:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829248.202534:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3615 00000100:00080000:1.0:1787829248.202540:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829248.202545:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829248.202551:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff9a7890b1e940 x1874671476929792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829248.202575:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829248.202623:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476930176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829248.202633:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476929792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829248.202638:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829248.202650:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829248.202655:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829248.202658:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829248.202661:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3615 00000100:00080000:1.0:1787829248.202669:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829248.202669:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829248.202678:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476929792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829248.202680:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829248.202691:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829248.202703:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476930304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829248.202720:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829248.202750:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829248.202763:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff9a7890b1de00 x1874671476929920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829248.202773:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476929920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829248.202781:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829248.202784:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829248.202787:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476929920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829248.202792:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829248.202806:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829248.202814:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff9a7890b1d680 x1874671476930048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829248.202823:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476930048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829248.202830:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829248.202832:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829248.202835:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829248.202840:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829248.202884:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829248.202893:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff9a7890b1da40 x1874671476930176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829248.202904:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476930176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829248.202913:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829248.202916:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829248.202919:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829248.202924:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829248.202937:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829248.202944:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829248/real 1787829248] req@ffff9a7890b1e580 x1874671476930304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829248.202954:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476930304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829248.202962:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829248.202964:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829248.202966:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829249.226060:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829250.249944:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829251.274021:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829252.298003:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829253.321979:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829253.322009:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829253.322019:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829253.322023:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829253.322034:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3620 00000100:00080000:1.0:1787829253.322045:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829253.322052:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829253.322095:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476930432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829253.322125:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829253.322129:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829253.322131:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829253.322136:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3620 00000100:00080000:1.0:1787829253.322141:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829253.322146:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829253.322159:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829253.322166:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476930432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829253.322182:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476930560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829253.322194:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829253.322197:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829253.322198:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829253.322203:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3620 00000100:00080000:1.0:1787829253.322207:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829253.322210:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829253.322215:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829253.322227:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476930688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829253.322235:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829253.322238:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829253.322240:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829253.322244:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3620 00000100:00080000:1.0:1787829253.322247:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829253.322250:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829253.322254:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829253.322265:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476930816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829253.322274:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829253.322277:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829253.322279:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829253.322283:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3620 00000100:00080000:1.0:1787829253.322287:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829253.322289:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829253.322293:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829253.322304:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476930944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829253.322312:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829253.322350:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829253.322371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829253.322401:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829254.346014:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829255.370012:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829256.393951:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829256.394087:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787829256.394124:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829256.394139:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787829256.394180:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787829256.394193:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787829256.394204:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787829256.394215:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829256.394235:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829256] req@ffff9a7890b1f0c0 x1874671476930944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829256.394269:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476930944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829256.394286:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829256.394293:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829256.394298:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930944:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829256.394307:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829256] req@ffff9a7890b1f840 x1874671476930560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829256.394341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476930560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829256.394353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829256.394357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829256.394362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829256.394371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829256] req@ffff9a7890b1fc00 x1874671476930688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829256.394410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476930688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829256.394417:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829256.394420:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829256.394424:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829256.394431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829256] req@ffff9a7890b1c000 x1874671476930816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829256.394441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476930816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829256.394449:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829256.394453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829256.394457:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476930816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829256.394464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829253/real 1787829256] req@ffff9a7890b1f480 x1874671476930432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829256.394480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476930432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829256.394490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829256.394494:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829256.394524:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476930432:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829257.417935:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829258.441970:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829258.441978:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829258.441981:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829258.441990:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3625 00000100:00080000:3.0:1787829258.441998:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.442004:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829258.442032:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476931072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829258.442056:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829258.442059:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829258.442061:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829258.442065:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3625 00000100:00080000:3.0:1787829258.442069:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.442073:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829258.442082:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829258.442096:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476931200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829258.442107:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476931072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829258.442111:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829258.442119:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829258.442121:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829258.442126:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3625 00000100:00080000:3.0:1787829258.442131:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.442135:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829258.442140:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829258.442155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476931328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829258.442166:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829258.442169:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829258.442171:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1787829258.442178:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787829258.442179:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3625 00000100:00080000:3.0:1787829258.442186:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.442191:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829258.442197:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829258.442217:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476931456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829258.442227:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829258.442232:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff9a78a6776d00 x1874671476931072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829258.442233:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829258.442238:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829258.442247:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3625 00000100:00080000:3.0:1787829258.442251:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829258.442255:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829258.442260:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829258.442277:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476931072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829258.442279:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476931584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829258.442303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829258.442310:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829258.442314:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476931072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829258.442342:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829258.442369:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829258.442386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff9a78a6775680 x1874671476931200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829258.442400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476931200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787829258.442404:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829258.442412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829258.442418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829258.442423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829258.442430:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829258.442458:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829258.442471:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff9a78a6775e00 x1874671476931328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829258.442501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476931328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829258.442510:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829258.442513:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829258.442517:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829258.442523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829258.442543:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829258.442555:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff9a78a6777840 x1874671476931456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829258.442567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476931456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829258.442576:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829258.442580:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829258.442583:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829258.442588:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829258.442605:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829258.442615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829258/real 1787829258] req@ffff9a78a6774f00 x1874671476931584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829258.442627:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476931584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829258.442637:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829258.442640:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829258.442643:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829259.465992:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829260.490058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829261.513970:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829262.537952:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829263.561953:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829263.561964:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829263.561969:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829263.561980:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3631 00000100:00080000:2.0:1787829263.561996:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829263.562004:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829263.562037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829263.562052:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476931712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829263.562079:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829263.562082:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829263.562084:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829263.562087:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3631 00000100:00080000:2.0:1787829263.562090:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829263.562093:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829263.562104:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829263.562114:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476931840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829263.562123:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476931712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829263.562125:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829263.562132:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829263.562135:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829263.562139:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3631 00000100:00080000:2.0:1787829263.562141:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829263.562144:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829263.562147:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829263.562156:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476931968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829263.562163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829263.562167:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829263.562168:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829263.562170:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3631 00000100:00080000:2.0:1787829263.562172:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829263.562174:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829263.562178:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829263.562185:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476932096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829263.562191:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829263.562193:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829263.562194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829263.562198:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3631 00000100:00080000:2.0:1787829263.562201:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829263.562204:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829263.562207:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787829263.562214:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829263.562217:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476932224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829263.562271:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829263/real 1787829263] req@ffff9a78841dbc00 x1874671476931712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829263.562287:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476931712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829263.562297:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829263.562300:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829263.562304:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476931712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829263.562326:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829263.562343:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829263.562352:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829263/real 1787829263] req@ffff9a78841db0c0 x1874671476931840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829263.562360:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476931840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829263.562366:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829263.562368:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829263.562370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829263.562374:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829263.562385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829263.562390:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829263/real 1787829263] req@ffff9a78841da1c0 x1874671476931968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829263.562398:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476931968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829263.562408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829263.562411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829263.562413:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476931968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829263.562416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829263.562427:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829263.562432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829263/real 1787829263] req@ffff9a78841d9a40 x1874671476932096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829263.562439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476932096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829263.562460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829263.562463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829263.562465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829263.562469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829263.562481:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829263.562488:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829263/real 1787829263] req@ffff9a78841d8780 x1874671476932224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829263.562495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476932224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829263.562502:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829263.562504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829263.562506:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829264.586026:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829265.609967:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829266.633945:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829267.658004:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829268.681949:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829268.681974:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829268.681981:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787829268.681987:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829268.681997:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3636 00000100:00080000:2.0:1787829268.682017:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829268.682030:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829268.682095:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476932352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829268.682136:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829268.682143:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829268.682147:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829268.682154:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3636 00000100:00080000:2.0:1787829268.682162:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829268.682171:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829268.682186:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476932352:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829268.682190:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829268.682225:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476932480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829268.682249:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829268.682260:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829268.682264:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829268.682276:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3636 00000800:00000100:3.0:1787829268.682281:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829268.682289:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829268.682297:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829268.682305:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829268.682326:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476932608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829268.682339:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829268.682344:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829268.682347:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829268.682358:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3636 00000100:00100000:3.0:1787829268.682362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829268/real 1787829268] req@ffff9a78841d9e00 x1874671476932352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829268.682365:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829268.682379:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829268.682392:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829268.682429:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476932736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829268.682434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476932352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829268.682468:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829268.682486:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829268.682491:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829268.682492:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829268.682502:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3636 00000100:00080000:2.0:1787829268.682509:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829268.682509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787829268.682523:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829268.682528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476932352:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829268.682534:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829268.682555:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476932864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829268.682569:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829268.682611:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829268.682627:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829268/real 1787829268] req@ffff9a78841da580 x1874671476932480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829268.682642:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476932480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829268.682652:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829268.682656:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829268.682659:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829268.682667:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829268.682687:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829268.682698:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829268/real 1787829268] req@ffff9a78841d8000 x1874671476932608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829268.682711:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476932608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829268.682729:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829268.682733:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829268.682737:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829268.682744:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829268.682764:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829268.682776:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829268/real 1787829268] req@ffff9a78841d83c0 x1874671476932736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829268.682789:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476932736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829268.682799:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829268.682803:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829268.682806:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829268.682824:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829268.682873:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829268.682886:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829268/real 1787829268] req@ffff9a78841db480 x1874671476932864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829268.682904:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476932864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829268.682914:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829268.682917:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829268.682920:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476932864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829269.706125:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829270.730035:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829271.753941:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829272.778009:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829273.801962:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829273.801972:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829273.801978:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829273.801989:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3641 00000100:00080000:2.0:1787829273.802000:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829273.802007:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829273.802037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829273.802045:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476932992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829273.802075:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829273.802079:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829273.802081:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829273.802086:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3641 00000100:00080000:2.0:1787829273.802091:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829273.802095:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829273.802105:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829273.802117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476932992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829273.802122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476933120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829273.802136:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829273.802141:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829273.802143:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829273.802150:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3641 00000100:00080000:2.0:1787829273.802154:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829273.802158:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829273.802163:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829273.802175:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476933248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829273.802185:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829273.802189:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829273.802191:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829273.802194:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3641 00000100:00080000:2.0:1787829273.802198:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829273.802203:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829273.802208:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829273.802213:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829273.802237:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476933376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829273.802252:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829273.802256:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829273.802259:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829273.802265:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3641 00000100:00080000:2.0:1787829273.802272:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829273.802276:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829273.802282:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829273.802288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829273/real 1787829273] req@ffff9a78841db840 x1874671476932992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829273.802300:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476933504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829273.802311:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476932992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829273.802338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829273.802344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829273.802355:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476932992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829273.802404:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829273.802426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829273.802436:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829273/real 1787829273] req@ffff9a78841d92c0 x1874671476933120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829273.802445:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476933120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829273.802456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829273.802459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829273.802461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829273.802466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829273.802478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829273.802483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829273/real 1787829273] req@ffff9a78841d9680 x1874671476933248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829273.802491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476933248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829273.802497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829273.802498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829273.802500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829273.802504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829273.802515:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829273.802521:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829273/real 1787829273] req@ffff9a78841d8b40 x1874671476933376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829273.802528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476933376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829273.802535:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829273.802537:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829273.802540:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829273.802544:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829273.802554:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829273.802560:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829273/real 1787829273] req@ffff9a78841da940 x1874671476933504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829273.802567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476933504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829273.802578:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829273.802581:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829273.802585:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829274.825973:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829275.850030:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829276.874011:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829277.897971:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829278.922032:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829278.922032:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829278.922046:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829278.922073:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829278.922088:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3646 00000100:00080000:1.0:1787829278.922101:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829278.922110:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829278.922158:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476933632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829278.922192:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829278.922198:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829278.922201:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829278.922206:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3646 00000100:00080000:1.0:1787829278.922211:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829278.922217:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829278.922221:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476933632:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787829278.922229:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829278.922246:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476933760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829278.922260:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829278.922264:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829278.922266:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829278.922271:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3646 00000100:00080000:1.0:1787829278.922286:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829278.922293:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829278.922293:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829278.922299:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829278.922313:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476933888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829278.922326:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829278.922330:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829278.922333:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829278.922358:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3646 00000100:00080000:1.0:1787829278.922364:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829278.922369:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829278.922371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829278/real 1787829278] req@ffff9a7890b1f480 x1874671476933632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829278.922376:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829278.922391:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476934016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829278.922395:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476933632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829278.922405:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829278.922409:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829278.922411:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829278.922412:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829278.922417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787829278.922418:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3646 00000100:00080000:1.0:1787829278.922422:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829278.922423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476933632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829278.922427:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829278.922432:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829278.922444:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476934144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829278.922452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829278.922483:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829278.922503:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829278/real 1787829278] req@ffff9a7890b1c000 x1874671476933760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829278.922522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476933760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829278.922530:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829278.922533:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829278.922536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829278.922543:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829278.922564:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829278.922591:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829278/real 1787829278] req@ffff9a7890b1fc00 x1874671476933888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829278.922610:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476933888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829278.922617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829278.922619:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829278.922622:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476933888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829278.922626:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829278.922639:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829278.922647:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829278/real 1787829278] req@ffff9a7890b1f840 x1874671476934016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829278.922662:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476934016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829278.922669:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829278.922671:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829278.922674:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829278.922688:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829278.922701:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829278.922709:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829278/real 1787829278] req@ffff9a7890b1f0c0 x1874671476934144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829278.922718:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476934144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829303 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829278.922726:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829278.922728:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829278.922730:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829279.945952:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829280.969991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829281.993960:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829283.017992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829284.041930:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787829284.041938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829284.041940:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829284.041946:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829284.041973:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3651 00000100:00080000:1.0:1787829284.041990:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829284.041997:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829284.042047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476934272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829284.042071:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829284.042077:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829284.042079:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829284.042082:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3651 00000100:00080000:1.0:1787829284.042085:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829284.042088:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829284.042096:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829284.042107:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476934400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829284.042117:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829284.042120:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829284.042125:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829284.042126:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476934272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829284.042148:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3651 00000100:00080000:1.0:1787829284.042152:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829284.042155:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829284.042159:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829284.042171:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476934528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829284.042179:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829284.042182:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829284.042183:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829284.042186:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3651 00000100:00080000:1.0:1787829284.042212:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829284.042216:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829284.042221:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829284.042227:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829284.042233:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476934656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829284.042244:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829284.042247:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829284.042248:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829284.042251:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3651 00000100:00080000:1.0:1787829284.042254:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829284.042257:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829284.042262:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829284.042273:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476934784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829284.042278:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9a7890b1e580 x1874671476934272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.042301:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476934272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.042361:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.042365:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829284.042368:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476934272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829284.042407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829284.042430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829284.042441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9a7890b1da40 x1874671476934400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.042452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476934400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.042459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.042461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829284.042464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829284.042469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829284.042482:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829284.042489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9a7890b1d680 x1874671476934528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.042498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476934528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.042504:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.042507:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829284.042510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829284.042514:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829284.042545:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829284.042551:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9a7890b1de00 x1874671476934656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.042559:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476934656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.042565:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.042567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829284.042570:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829284.042574:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829284.042584:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829284.042590:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829284/real 1787829284] req@ffff9a7890b1e940 x1874671476934784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829284.042598:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476934784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829284.042616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829284.042618:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829284.042620:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476934784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829285.065925:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829286.089932:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829287.113933:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829288.137937:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829289.161939:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829289.161950:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829289.161954:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829289.161964:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3656 00000100:00080000:1.0:1787829289.161975:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829289.161983:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829289.162000:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829289.162022:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476934912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829289.162050:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829289.162054:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829289.162063:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829289.162068:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3656 00000100:00080000:1.0:1787829289.162073:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829289.162078:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829289.162088:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829289.162091:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476934912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829289.162104:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476935040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829289.162117:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829289.162120:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829289.162122:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829289.162127:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3656 00000100:00080000:1.0:1787829289.162146:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829289.162151:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829289.162156:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829289.162160:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829289.162177:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476935168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829289.162189:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829289.162193:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829289.162196:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829289.162199:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff9a7890b1e940 x1874671476934912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829289.162204:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3656 00000100:00080000:1.0:1787829289.162211:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829289.162218:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829289.162223:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476934912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829289.162226:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787829289.162238:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.162241:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829289.162248:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476934912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829289.162249:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476935296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829289.162283:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829289.162287:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829289.162289:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829289.162294:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3656 00000100:00080000:1.0:1787829289.162297:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829289.162301:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829289.162306:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829289.162319:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476935424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829289.162343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829289.162364:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829289.162374:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff9a7890b1de00 x1874671476935040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829289.162383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476935040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829289.162389:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.162391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829289.162393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829289.162397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829289.162408:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829289.162414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff9a7890b1d680 x1874671476935168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829289.162425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476935168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829289.162431:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.162432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829289.162434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829289.162443:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829289.162453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829289.162459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff9a7890b1da40 x1874671476935296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829289.162466:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476935296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829289.162472:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.162473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829289.162475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829289.162479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829289.162488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829289.162493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829289/real 1787829289] req@ffff9a7890b1e580 x1874671476935424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829289.162500:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476935424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829289.162510:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829289.162512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829289.162514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829290.185968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829291.209974:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829292.233965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829293.257977:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829294.281963:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829294.281963:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829294.281982:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829294.281987:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.281997:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3661 00000100:00080000:2.0:1787829294.282008:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.282015:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829294.282051:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476935552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829294.282075:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829294.282079:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829294.282080:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.282083:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3661 00000100:00080000:2.0:1787829294.282087:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.282090:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829294.282099:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829294.282107:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476935680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829294.282119:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829294.282120:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476935552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829294.282138:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829294.282142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.282149:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3661 00000100:00080000:2.0:1787829294.282154:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.282160:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829294.282166:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829294.282186:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476935808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829294.282193:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829294.282196:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829294.282199:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.282202:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3661 00000100:00080000:2.0:1787829294.282206:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.282208:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829294.282215:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829294.282217:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829294.282230:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476935936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829294.282253:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829294.282255:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829294.282256:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829294.282260:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3661 00000100:00080000:2.0:1787829294.282264:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829294.282267:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829294.282289:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829294.282297:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9a78841da940 x1874671476935552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829294.282307:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476936064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829294.282326:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476935552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.282340:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.282343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829294.282349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476935552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829294.282394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829294.282421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829294.282435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9a78841d8b40 x1874671476935680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829294.282446:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476935680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.282452:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.282455:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829294.282458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829294.282463:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829294.282478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829294.282485:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9a78841d9680 x1874671476935808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829294.282493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476935808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.282498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.282500:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829294.282502:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829294.282506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829294.282517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829294.282523:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9a78841d92c0 x1874671476935936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829294.282530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476935936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.282537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.282539:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829294.282541:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476935936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829294.282545:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829294.282555:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829294.282561:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829294/real 1787829294] req@ffff9a78841db840 x1874671476936064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829294.282569:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476936064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829294.282575:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829294.282577:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829294.282579:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829295.306097:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829296.329925:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829297.354005:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829298.378049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829299.401928:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829299.401940:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829299.401947:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829299.401960:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3666 00010000:00010000:3.0:1787829299.401965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829299.401974:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829299.401984:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829299.402031:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476936192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829299.402061:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829299.402065:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829299.402067:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829299.402071:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3666 00000100:00080000:2.0:1787829299.402074:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829299.402078:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829299.402090:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829299.402102:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476936192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829299.402104:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476936320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829299.402120:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829299.402123:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829299.402124:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829299.402128:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3666 00000100:00080000:2.0:1787829299.402131:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829299.402134:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829299.402154:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829299.402166:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476936448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829299.402181:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829299.402184:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829299.402186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829299.402189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3666 00000100:00080000:2.0:1787829299.402193:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829299.402210:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829299.402216:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829299.402220:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829299.402230:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476936576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829299.402250:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829299.402254:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829299.402256:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829299.402260:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3666 00000100:00080000:2.0:1787829299.402264:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829299.402266:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829299.402270:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829299.402280:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476936704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829299.402310:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff9a78841db480 x1874671476936192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.402330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476936192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.402340:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.402344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829299.402348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476936192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829299.402375:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829299.402397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829299.402409:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff9a78841d83c0 x1874671476936320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.402419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476936320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.402427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.402429:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829299.402432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829299.402436:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829299.402449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829299.402456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff9a78841d8000 x1874671476936448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.402476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476936448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.402491:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.402493:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829299.402496:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829299.402501:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829299.402515:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829299.402523:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff9a78841da580 x1874671476936576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.402532:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476936576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.402558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.402561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829299.402563:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829299.402567:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829299.402579:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829299.402585:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829299/real 1787829299] req@ffff9a78841d9e00 x1874671476936704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829299.402594:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476936704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829299.402602:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829299.402605:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829299.402607:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829300.425995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829301.450005:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829302.473962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829303.498014:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829304.521973:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829304.521986:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829304.521991:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829304.522003:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3672 00000100:00080000:2.0:1787829304.522018:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829304.522026:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829304.522040:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829304.522071:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476936832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829304.522096:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829304.522099:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829304.522100:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829304.522103:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3672 00000100:00080000:2.0:1787829304.522106:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829304.522108:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829304.522119:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829304.522129:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476936960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829304.522137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829304.522139:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829304.522142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829304.522146:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476936832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829304.522148:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3672 00000100:00080000:2.0:1787829304.522155:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829304.522158:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829304.522175:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829304.522190:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476937088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829304.522213:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829304.522217:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829304.522220:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829304.522225:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3672 00000100:00080000:2.0:1787829304.522246:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829304.522250:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829304.522256:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829304.522261:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829304.522270:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476937216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829304.522280:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829304.522283:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829304.522284:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829304.522288:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3672 00000100:00080000:2.0:1787829304.522291:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829304.522294:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829304.522297:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829304.522306:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476937344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829304.522321:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829304/real 1787829304] req@ffff9a78841d8780 x1874671476936832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829304.522349:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476936832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829304.522358:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829304.522370:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829304.522374:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476936832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829304.522397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829304.522426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829304.522435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829304/real 1787829304] req@ffff9a78841d9a40 x1874671476936960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829304.522444:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476936960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829304.522450:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829304.522453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829304.522455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476936960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829304.522459:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829304.522470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829304.522476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829304/real 1787829304] req@ffff9a78841da1c0 x1874671476937088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829304.522484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476937088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829304.522489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829304.522491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829304.522493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829304.522497:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829304.522507:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829304.522513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829304/real 1787829304] req@ffff9a78841db0c0 x1874671476937216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829304.522521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476937216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829304.522528:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829304.522530:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829304.522532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829304.522536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829304.522546:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829304.522551:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829304/real 1787829304] req@ffff9a78841dbc00 x1874671476937344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829304.522559:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476937344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829304.522566:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829304.522568:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829304.522569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829305.545959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829306.569981:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829307.593968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829308.618083:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829309.641949:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829309.641960:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829309.641964:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829309.641977:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3677 00000100:00080000:2.0:1787829309.641990:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829309.641997:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829309.642006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829309.642043:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476937472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829309.642066:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829309.642069:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829309.642071:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829309.642074:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3677 00000100:00080000:2.0:1787829309.642076:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829309.642079:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829309.642090:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829309.642099:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476937600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829309.642107:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829309.642111:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829309.642115:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829309.642117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476937472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829309.642123:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3677 00000100:00080000:2.0:1787829309.642138:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829309.642144:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829309.642150:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829309.642166:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476937728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829309.642177:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829309.642181:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829309.642183:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829309.642189:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3677 00000100:00080000:2.0:1787829309.642204:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829309.642209:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829309.642215:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829309.642222:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829309.642233:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476937856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829309.642245:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829309.642248:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829309.642251:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829309.642256:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3677 00000100:00080000:2.0:1787829309.642259:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829309.642264:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829309.642270:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829309.642287:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829309/real 1787829309] req@ffff9a78841dbc00 x1874671476937472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829309.642293:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476937984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829309.642338:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476937472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829309.642352:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829309.642357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829309.642362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476937472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829309.642389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829309.642414:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829309.642429:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829309/real 1787829309] req@ffff9a78841db0c0 x1874671476937600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829309.642443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476937600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829309.642454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829309.642457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829309.642461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829309.642467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829309.642488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829309.642501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829309/real 1787829309] req@ffff9a78841da1c0 x1874671476937728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829309.642514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476937728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829309.642530:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829309.642534:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829309.642537:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829309.642544:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829309.642564:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829309.642576:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829309/real 1787829309] req@ffff9a78841d9a40 x1874671476937856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829309.642589:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476937856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829309.642600:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829309.642603:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829309.642607:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829309.642613:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829309.642633:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829309.642645:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829309/real 1787829309] req@ffff9a78841d8780 x1874671476937984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829309.642657:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476937984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829309.642667:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829309.642671:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829309.642675:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476937984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829310.665979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829311.689909:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829312.714025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829313.738000:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829314.761906:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829314.761914:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829314.761918:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829314.761927:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3682 00000100:00080000:2.0:1787829314.761936:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829314.761943:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787829314.761970:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829314.761984:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476938112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829314.762012:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829314.762018:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829314.762019:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829314.762023:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3682 00000100:00080000:2.0:1787829314.762026:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829314.762029:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829314.762039:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829314.762047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476938240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829314.762057:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476938112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829314.762061:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829314.762068:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829314.762070:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829314.762074:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3682 00000100:00080000:2.0:1787829314.762076:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829314.762079:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829314.762082:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829314.762106:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476938368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829314.762113:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829314.762116:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829314.762117:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829314.762120:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3682 00000100:00080000:2.0:1787829314.762144:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829314.762147:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829314.762151:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829314.762162:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476938496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829314.762164:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829314.762177:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829314.762180:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829314.762183:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829314.762185:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3682 00000100:00080000:2.0:1787829314.762188:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829314.762191:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829314.762194:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829314.762201:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476938624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829314.762242:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829314/real 1787829314] req@ffff9a78841d9e00 x1874671476938112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829314.762260:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476938112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829314.762269:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829314.762273:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829314.762276:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476938112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829314.762302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829314.762321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829314.762330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829314/real 1787829314] req@ffff9a78841da580 x1874671476938240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829314.762338:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476938240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829314.762343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829314.762345:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829314.762348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829314.762352:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829314.762363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829314.762369:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829314/real 1787829314] req@ffff9a78841d8000 x1874671476938368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829314.762376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476938368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829314.762382:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829314.762383:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829314.762385:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829314.762389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829314.762399:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829314.762404:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829314/real 1787829314] req@ffff9a78841d83c0 x1874671476938496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829314.762411:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476938496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829314.762419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829314.762421:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829314.762423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829314.762427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829314.762436:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829314.762442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829314/real 1787829314] req@ffff9a78841db480 x1874671476938624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829314.762454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476938624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829314.762461:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829314.762463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829314.762465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829315.785968:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829316.809973:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829317.833962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829318.857970:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829319.881958:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829319.881968:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829319.881974:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829319.881986:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3687 00010000:00010000:0.0:1787829319.881990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829319.881998:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829319.882007:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829319.882046:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476938752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829319.882082:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829319.882085:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829319.882086:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829319.882090:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3687 00000100:00080000:1.0:1787829319.882093:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829319.882101:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829319.882110:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829319.882118:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476938752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829319.882124:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476938880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829319.882137:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829319.882141:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829319.882143:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829319.882148:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3687 00000100:00080000:1.0:1787829319.882152:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829319.882157:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829319.882162:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829319.882177:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476939008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829319.882189:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829319.882192:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829319.882194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829319.882199:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3687 00000100:00080000:1.0:1787829319.882204:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829319.882209:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829319.882215:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829319.882232:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476939136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829319.882232:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938880:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829319.882248:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829319.882251:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829319.882253:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829319.882258:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3687 00000100:00080000:1.0:1787829319.882263:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829319.882270:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939008:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829319.882271:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829319.882283:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829319.882300:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476939264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829319.882311:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829319.882346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829320.905929:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829321.930116:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829322.953999:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829322.954070:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787829322.954112:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829322.954128:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829322.954167:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829322.954178:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829322.954189:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829322.954201:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829322.954212:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829319/real 1787829322] req@ffff9a7890b1c000 x1874671476939136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829322.954239:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476939136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829322.954252:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829322.954256:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829322.954259:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829322.954269:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829319/real 1787829322] req@ffff9a7890b1f840 x1874671476938880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829322.954282:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476938880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829322.954298:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829322.954302:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829322.954305:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476938880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829322.954311:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829319/real 1787829322] req@ffff9a7890b1fc00 x1874671476939008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829322.954324:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476939008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829322.954331:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829322.954334:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829322.954338:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829322.954344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829319/real 1787829322] req@ffff9a7890b1f0c0 x1874671476938752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829322.954357:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476938752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829322.954365:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829322.954369:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829322.954372:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476938752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829322.954399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829319/real 1787829322] req@ffff9a7890b1f480 x1874671476939264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829322.954413:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476939264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829322.954424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829322.954427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829322.954431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829323.977969:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829325.001950:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829325.001963:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829325.001968:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829325.001982:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3692 00010000:00010000:0.0:1787829325.001985:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829325.001997:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829325.002008:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829325.002072:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476939392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829325.002104:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829325.002109:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829325.002111:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829325.002116:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3692 00000100:00080000:1.0:1787829325.002121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829325.002127:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829325.002139:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476939392:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787829325.002141:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829325.002161:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476939520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829325.002179:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829325.002183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829325.002185:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829325.002190:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3692 00000100:00080000:1.0:1787829325.002209:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829325.002214:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829325.002220:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829325.002224:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829325.002236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476939648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829325.002247:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829325.002251:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829325.002253:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829325.002257:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3692 00000100:00080000:1.0:1787829325.002261:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829325.002266:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829325.002272:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829325.002285:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476939776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829325.002296:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9a7890b1f480 x1874671476939392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829325.002298:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829325.002307:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829325.002311:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829325.002320:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3692 00000100:00080000:1.0:1787829325.002326:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829325.002332:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476939392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829325.002333:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829325.002349:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787829325.002354:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829325.002356:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829325.002363:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476939392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829325.002376:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476939904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829325.002400:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829325.002430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829325.002444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9a7890b1f0c0 x1874671476939520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829325.002465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476939520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829325.002474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829325.002478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829325.002482:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829325.002488:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829325.002510:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829325.002522:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9a7890b1fc00 x1874671476939648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829325.002535:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476939648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829325.002543:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829325.002546:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829325.002549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829325.002556:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829325.002576:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829325.002588:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9a7890b1f840 x1874671476939776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829325.002601:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476939776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829325.002619:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829325.002623:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829325.002627:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829325.002633:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829325.002652:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829325.002663:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9a7890b1c000 x1874671476939904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829325.002675:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476939904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829325.002686:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829325.002689:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829325.002692:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476939904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829326.026079:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829327.049916:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829328.074024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829329.097972:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829330.121973:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787829330.121977:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829330.121999:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829330.122005:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829330.122016:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3697 00000100:00080000:2.0:1787829330.122027:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829330.122035:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829330.122079:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476940032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829330.122103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829330.122106:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829330.122107:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829330.122110:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3697 00000100:00080000:2.0:1787829330.122113:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829330.122116:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829330.122126:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829330.122140:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476940160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829330.122140:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476940032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829330.122157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829330.122160:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829330.122162:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829330.122168:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3697 00000100:00080000:2.0:1787829330.122172:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829330.122176:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829330.122182:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829330.122199:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476940288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829330.122208:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829330.122211:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829330.122216:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829330.122216:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829330.122223:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3697 00000100:00080000:2.0:1787829330.122229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829330.122232:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829330.122237:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829330.122246:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476940416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829330.122257:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829330.122260:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829330.122262:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829330.122267:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3697 00000100:00080000:2.0:1787829330.122271:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829330.122276:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829330.122282:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829330.122297:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476940544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829330.122302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9a78841db840 x1874671476940032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829330.122329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476940032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829330.122340:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829330.122344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829330.122347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476940032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829330.122371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829330.122391:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829330.122401:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9a78841d92c0 x1874671476940160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829330.122410:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476940160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829330.122416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829330.122418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829330.122420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829330.122424:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829330.122451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829330.122463:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9a78841d9680 x1874671476940288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829330.122477:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476940288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829330.122486:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829330.122490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829330.122494:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829330.122501:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829330.122522:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829330.122541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9a78841d8b40 x1874671476940416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829330.122550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476940416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829330.122557:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829330.122559:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829330.122562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829330.122566:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829330.122580:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829330.122586:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9a78841da940 x1874671476940544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829330.122593:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476940544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829330.122600:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829330.122601:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829330.122603:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829331.145962:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829332.170040:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829333.193911:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829334.218031:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829335.241938:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829335.241958:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829335.241963:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829335.241972:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3702 00010000:00010000:0.0:1787829335.241978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829335.241983:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829335.241991:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829335.242025:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476940672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829335.242052:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829335.242055:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829335.242056:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829335.242059:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3702 00000100:00080000:2.0:1787829335.242062:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829335.242065:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829335.242073:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829335.242083:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476940800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829335.242091:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829335.242098:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829335.242099:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476940672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829335.242100:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829335.242107:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3702 00000100:00080000:2.0:1787829335.242114:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829335.242119:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829335.242122:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829335.242134:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476940928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829335.242141:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829335.242143:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829335.242144:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829335.242147:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3702 00000100:00080000:2.0:1787829335.242149:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829335.242152:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829335.242156:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829335.242163:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476941056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829335.242172:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829335.242175:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829335.242177:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829335.242184:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3702 00000100:00080000:2.0:1787829335.242188:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829335.242193:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829335.242201:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829335.242206:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829335.242224:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476941184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829335.242275:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9a78841da940 x1874671476940672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829335.242299:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476940672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829335.242309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829335.242312:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829335.242315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476940672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829335.242341:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829335.242360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829335.242370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9a78841d8b40 x1874671476940800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829335.242379:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476940800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829335.242385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829335.242387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829335.242389:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829335.242393:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829335.242404:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829335.242409:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9a78841d9680 x1874671476940928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829335.242417:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476940928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829335.242427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829335.242430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829335.242432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476940928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829335.242435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829335.242446:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829335.242452:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9a78841d92c0 x1874671476941056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829335.242459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476941056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829335.242466:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829335.242468:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829335.242470:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829335.242474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829335.242484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829335.242489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9a78841db840 x1874671476941184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829335.242497:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476941184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829335.242503:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829335.242505:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829335.242507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829336.265938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829337.290012:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829338.314014:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829339.338001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829340.361981:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787829340.361989:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829340.361989:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829340.361996:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.362005:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3707 00000100:00080000:1.0:1787829340.362014:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.362019:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829340.362050:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476941312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829340.362072:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829340.362074:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829340.362076:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.362078:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3707 00000100:00080000:1.0:1787829340.362080:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.362083:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829340.362091:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829340.362103:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476941440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829340.362104:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476941312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829340.362118:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829340.362122:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829340.362124:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.362127:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3707 00000100:00080000:1.0:1787829340.362130:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.362132:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829340.362135:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829340.362143:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476941568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829340.362150:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829340.362152:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829340.362154:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.362156:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3707 00000100:00080000:1.0:1787829340.362158:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.362161:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829340.362166:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829340.362168:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829340.362178:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476941696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829340.362185:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829340.362187:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829340.362189:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829340.362191:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3707 00000100:00080000:1.0:1787829340.362193:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829340.362196:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829340.362199:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829340.362208:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476941824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829340.362216:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829340] req@ffff9a7890b1e580 x1874671476941312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829340.362232:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476941312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829340.362241:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829340.362244:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829340.362247:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476941312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829340.362266:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829340.362281:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829340.362290:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829340] req@ffff9a7890b1da40 x1874671476941440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829340.362298:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476941440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829340.362304:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829340.362306:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829340.362308:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829340.362312:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829340.362322:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829340.362327:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829340] req@ffff9a7890b1d680 x1874671476941568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829340.362334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476941568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829340.362339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829340.362341:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829340.362343:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829340.362347:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829340.362356:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829340.362362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829340] req@ffff9a7890b1de00 x1874671476941696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829340.362369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476941696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829340.362375:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829340.362376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829340.362378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829340.362381:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829340.362390:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829340.362395:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829340/real 1787829340] req@ffff9a7890b1e940 x1874671476941824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829340.362407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476941824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829340.362413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829340.362415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829340.362417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476941824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829341.385993:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829342.409931:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829343.434000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829344.457962:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829345.481928:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829345.481936:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829345.481939:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829345.481949:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3712 00010000:00010000:3.0:1787829345.481955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829345.481958:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829345.481966:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829345.481999:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476941952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829345.482021:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829345.482023:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829345.482025:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829345.482028:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3712 00000100:00080000:1.0:1787829345.482030:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829345.482033:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829345.482041:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829345.482051:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476941952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829345.482052:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476942080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829345.482065:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829345.482067:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829345.482068:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829345.482071:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3712 00000100:00080000:1.0:1787829345.482074:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829345.482076:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829345.482080:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829345.482088:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476942208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829345.482095:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829345.482097:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829345.482099:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829345.482101:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3712 00000100:00080000:1.0:1787829345.482104:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829345.482106:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829345.482109:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829345.482110:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829345.482119:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476942336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829345.482127:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829345.482130:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829345.482131:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829345.482134:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3712 00000100:00080000:1.0:1787829345.482136:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829345.482138:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829345.482141:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829345.482149:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476942464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829345.482160:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829345/real 1787829345] req@ffff9a7890b1e940 x1874671476941952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829345.482175:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476941952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829345.482183:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829345.482186:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829345.482189:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476941952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829345.482208:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829345.482223:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829345.482232:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829345/real 1787829345] req@ffff9a7890b1de00 x1874671476942080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829345.482240:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476942080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829345.482246:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829345.482248:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829345.482251:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829345.482254:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829345.482265:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829345.482272:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829345/real 1787829345] req@ffff9a7890b1d680 x1874671476942208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829345.482284:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476942208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829345.482293:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829345.482296:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829345.482300:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829345.482307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829345.482327:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829345.482339:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829345/real 1787829345] req@ffff9a7890b1da40 x1874671476942336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829345.482352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476942336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829345.482359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829345.482361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829345.482364:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829345.482368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829345.482381:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829345.482387:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829345/real 1787829345] req@ffff9a7890b1e580 x1874671476942464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829345.482395:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476942464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829345.482401:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829345.482402:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829345.482404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829346.505961:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829347.530015:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829348.553950:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829349.577998:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829350.601908:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829350.601920:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829350.601925:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829350.601937:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3718 00000100:00080000:1.0:1787829350.601950:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829350.601957:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829350.601967:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829350.602000:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476942592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829350.602023:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829350.602026:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829350.602028:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829350.602031:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3718 00000100:00080000:1.0:1787829350.602033:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829350.602037:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829350.602047:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829350.602059:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476942720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829350.602062:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476942592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829350.602075:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829350.602078:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829350.602079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829350.602083:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3718 00000100:00080000:1.0:1787829350.602084:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829350.602087:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829350.602091:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829350.602102:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476942848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829350.602117:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829350.602126:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829350.602128:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829350.602134:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3718 00000100:00080000:1.0:1787829350.602138:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829350.602142:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829350.602148:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829350.602163:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476942976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829350.602167:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829350.602179:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829350.602182:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829350.602184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829350.602189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3718 00000100:00080000:1.0:1787829350.602199:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829350.602202:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829350.602206:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829350.602216:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476943104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829350.602245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829350/real 1787829350] req@ffff9a7890b1c000 x1874671476942592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829350.602264:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476942592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829350.602273:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829350.602277:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829350.602281:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476942592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829350.602305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829350.602327:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829350.602337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829350/real 1787829350] req@ffff9a7890b1f840 x1874671476942720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829350.602345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476942720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829350.602352:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829350.602354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829350.602356:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829350.602360:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829350.602370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829350.602376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829350/real 1787829350] req@ffff9a7890b1fc00 x1874671476942848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829350.602384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476942848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829350.602389:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829350.602391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829350.602393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829350.602396:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829350.602406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829350.602411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829350/real 1787829350] req@ffff9a7890b1f0c0 x1874671476942976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829350.602424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476942976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829350.602432:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829350.602433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829350.602435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476942976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829350.602439:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829350.602455:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829350.602473:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829350/real 1787829350] req@ffff9a7890b1f480 x1874671476943104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829350.602486:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476943104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829350.602497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829350.602501:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829350.602504:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829351.625961:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829352.649937:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829353.673961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829354.697892:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829355.721915:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829355.721923:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829355.721928:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829355.721937:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3723 00000100:00080000:1.0:1787829355.721950:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829355.721957:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829355.721980:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829355.721992:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476943232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829355.722011:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829355.722014:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829355.722015:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829355.722018:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3723 00000100:00080000:1.0:1787829355.722020:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829355.722022:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829355.722030:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829355.722039:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476943360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829355.722046:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829355.722049:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829355.722050:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829355.722056:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3723 00000100:00100000:3.0:1787829355.722056:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476943232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829355.722062:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829355.722069:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829355.722075:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829355.722089:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476943488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829355.722106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829355.722110:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829355.722112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829355.722116:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3723 00000100:00080000:1.0:1787829355.722121:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829355.722126:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829355.722129:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829355.722135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829355.722147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476943616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829355.722153:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829355.722155:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829355.722157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829355.722159:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3723 00000100:00080000:1.0:1787829355.722163:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829355.722165:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829355.722168:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829355.722177:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476943744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829355.722186:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829355/real 1787829355] req@ffff9a7890b1f480 x1874671476943232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829355.722206:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476943232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829355.722218:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829355.722222:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829355.722226:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476943232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829355.722249:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829355.722272:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829355.722285:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829355/real 1787829355] req@ffff9a7890b1f0c0 x1874671476943360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829355.722298:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476943360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829355.722306:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829355.722310:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829355.722313:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829355.722319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829355.722340:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829355.722351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829355/real 1787829355] req@ffff9a7890b1fc00 x1874671476943488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829355.722363:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476943488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829355.722372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829355.722375:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829355.722379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829355.722385:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829355.722406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829355.722419:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829355/real 1787829355] req@ffff9a7890b1f840 x1874671476943616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829355.722431:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476943616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829355.722440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829355.722444:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829355.722447:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829355.722454:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829355.722474:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829355.722485:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829355/real 1787829355] req@ffff9a7890b1c000 x1874671476943744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829355.722498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476943744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829355.722507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829355.722511:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829355.722514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476943744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829356.746021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829357.769935:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829358.794027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829359.817979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829360.841930:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829360.841938:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829360.841942:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829360.841950:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3728 00000100:00080000:1.0:1787829360.841960:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787829360.841965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829360.841966:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829360.841999:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476943872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829360.842020:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829360.842027:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829360.842029:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829360.842032:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3728 00000100:00080000:1.0:1787829360.842035:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829360.842039:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829360.842046:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829360.842050:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476943872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829360.842058:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476944000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829360.842067:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829360.842069:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829360.842071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829360.842074:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3728 00000100:00080000:1.0:1787829360.842076:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829360.842080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829360.842086:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829360.842099:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476944128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829360.842107:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829360.842117:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829360.842122:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829360.842124:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829360.842129:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3728 00000100:00080000:1.0:1787829360.842134:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829360.842138:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829360.842143:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829360.842149:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829360/real 1787829360] req@ffff9a7890b1e580 x1874671476943872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829360.842155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476944256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829360.842168:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829360.842172:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787829360.842173:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476943872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829360.842174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829360.842183:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3728 00000100:00080000:1.0:1787829360.842188:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829360.842192:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829360.842192:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787829360.842200:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829360.842201:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829360.842208:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476943872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829360.842212:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476944384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829360.842237:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829360.842254:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829360.842263:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829360/real 1787829360] req@ffff9a7890b1da40 x1874671476944000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829360.842271:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476944000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829360.842276:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829360.842279:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829360.842283:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829360.842289:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829360.842308:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829360.842320:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829360/real 1787829360] req@ffff9a7890b1d680 x1874671476944128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829360.842348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476944128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829360.842354:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829360.842356:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829360.842358:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829360.842363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829360.842378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829360.842386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829360/real 1787829360] req@ffff9a7890b1de00 x1874671476944256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829360.842393:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476944256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829360.842402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829360.842405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829360.842409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829360.842415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829360.842433:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829360.842445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829360/real 1787829360] req@ffff9a7890b1e940 x1874671476944384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829360.842456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476944384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829360.842467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829360.842469:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829360.842471:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829361.865956:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829362.889975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829363.913927:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829364.937997:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829365.961949:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829365.961957:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829365.961961:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829365.961970:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3733 00000100:00080000:3.0:1787829365.961978:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829365.961983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829365.961984:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829365.962018:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476944512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829365.962040:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829365.962044:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829365.962047:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829365.962050:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3733 00000100:00080000:3.0:1787829365.962054:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829365.962058:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829365.962067:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829365.962081:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476944640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829365.962101:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476944512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829365.962102:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829365.962107:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829365.962109:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829365.962114:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3733 00000100:00080000:3.0:1787829365.962117:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829365.962120:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829365.962124:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829365.962136:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476944768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829365.962143:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829365.962145:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829365.962147:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829365.962149:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3733 00000100:00080000:3.0:1787829365.962160:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829365.962162:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829365.962165:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829365.962172:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829365.962175:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476944896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829365.962182:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829365.962185:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829365.962187:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829365.962190:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3733 00000100:00080000:3.0:1787829365.962195:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829365.962197:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829365.962200:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829365.962209:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476945024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829365.962227:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829365/real 1787829365] req@ffff9a78a6777c00 x1874671476944512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829365.962246:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476944512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829365.962257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829365.962262:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829365.962266:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476944512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829365.962289:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829365.962312:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829365.962326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829365/real 1787829365] req@ffff9a78a67743c0 x1874671476944640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829365.962341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476944640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829365.962351:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829365.962355:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829365.962359:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829365.962364:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829365.962380:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829365.962388:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829365/real 1787829365] req@ffff9a78a6774f00 x1874671476944768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829365.962396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476944768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829365.962402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829365.962404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829365.962406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829365.962410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829365.962420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829365.962426:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829365/real 1787829365] req@ffff9a78a6777840 x1874671476944896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829365.962434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476944896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829365.962440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829365.962442:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829365.962444:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476944896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829365.962447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829365.962461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829365.962472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829365/real 1787829365] req@ffff9a78a6775e00 x1874671476945024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829365.962484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476945024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829390 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829365.962495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829365.962498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829365.962502:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829366.986002:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829368.009970:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829369.033982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829370.057907:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829371.081938:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829371.081946:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829371.081950:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829371.081959:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3738 00000100:00080000:1.0:1787829371.081968:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829371.081973:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829371.081995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829371.082007:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476945152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829371.082030:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829371.082033:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829371.082034:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829371.082037:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3738 00000100:00080000:1.0:1787829371.082039:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829371.082041:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829371.082049:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829371.082059:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476945280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829371.082066:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829371.082069:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829371.082070:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829371.082073:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3738 00000100:00080000:1.0:1787829371.082074:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829371.082077:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829371.082080:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829371.082088:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476945408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829371.082090:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476945152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829371.082099:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829371.082103:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829371.082104:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829371.082106:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3738 00000100:00080000:1.0:1787829371.082109:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829371.082111:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829371.082115:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829371.082122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476945536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829371.082127:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829371.082130:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829371.082131:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829371.082134:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3738 00000100:00080000:1.0:1787829371.082147:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829371.082149:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829371.082152:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829371.082160:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829371.082161:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476945664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829371.082209:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9a7890b1e940 x1874671476945152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829371.082228:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476945152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829371.082239:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829371.082251:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829371.082256:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476945152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829371.082264:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829371.082289:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829371.082302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9a7890b1de00 x1874671476945280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829371.082316:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476945280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829371.082324:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829371.082328:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829371.082332:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829371.082355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829371.082378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829371.082391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9a7890b1d680 x1874671476945408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829371.082404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476945408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829371.082413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829371.082416:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829371.082420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829371.082427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829371.082448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829371.082461:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9a7890b1da40 x1874671476945536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829371.082473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476945536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829371.082483:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829371.082486:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829371.082489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829371.082496:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829371.082516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829371.082529:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829371/real 1787829371] req@ffff9a7890b1e580 x1874671476945664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829371.082542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476945664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829371.082552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829371.082555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829371.082559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829372.105947:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829373.129944:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829374.154069:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829375.177959:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829376.201947:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829376.201961:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829376.201967:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787829376.201981:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829376.201982:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3743 00000100:00080000:1.0:1787829376.201998:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829376.202009:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829376.202058:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476945792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829376.202084:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829376.202090:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829376.202091:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829376.202094:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3743 00000100:00080000:1.0:1787829376.202098:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829376.202101:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829376.202111:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829376.202122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476945920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829376.202135:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829376.202138:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829376.202140:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829376.202146:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3743 00000100:00080000:1.0:1787829376.202150:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829376.202154:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829376.202160:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829376.202176:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476946048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829376.202184:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476945792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829376.202191:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829376.202199:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829376.202202:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829376.202207:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3743 00000100:00080000:1.0:1787829376.202212:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829376.202217:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829376.202225:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829376.202243:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476946176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829376.202251:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829376.202254:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829376.202256:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829376.202259:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3743 00000100:00080000:1.0:1787829376.202270:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829376.202273:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829376.202277:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829376.202284:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829376.202291:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476946304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829376.202346:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff9a7890b1c000 x1874671476945792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829376.202368:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476945792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829376.202379:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829376.202385:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829376.202389:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476945792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829376.202399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829376.202427:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829376.202441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff9a7890b1f840 x1874671476945920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829376.202466:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476945920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829376.202476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829376.202479:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829376.202483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476945920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829376.202511:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829376.202531:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829376.202540:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff9a7890b1fc00 x1874671476946048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829376.202548:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476946048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829376.202559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829376.202561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829376.202563:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829376.202568:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829376.202580:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829376.202586:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff9a7890b1f0c0 x1874671476946176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829376.202593:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476946176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829376.202600:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829376.202602:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829376.202604:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829376.202608:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829376.202617:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829376.202623:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829376/real 1787829376] req@ffff9a7890b1f480 x1874671476946304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829376.202630:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476946304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829376.202636:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829376.202638:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829376.202640:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829377.226008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829378.250025:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829379.273969:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829380.297995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829381.321901:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829381.321910:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829381.321915:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829381.321925:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3748 00000100:00080000:1.0:1787829381.321936:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829381.321942:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829381.321961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829381.321984:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476946432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829381.322005:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829381.322011:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829381.322012:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829381.322015:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3748 00000100:00080000:1.0:1787829381.322019:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829381.322022:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829381.322031:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829381.322045:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476946432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829381.322047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476946560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829381.322063:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829381.322067:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829381.322069:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829381.322074:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3748 00000100:00080000:1.0:1787829381.322098:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829381.322101:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829381.322106:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829381.322106:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829381.322122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476946688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829381.322130:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829381.322132:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829381.322134:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829381.322137:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3748 00000100:00080000:1.0:1787829381.322139:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829381.322142:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829381.322147:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829381.322157:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff9a7890b1f480 x1874671476946432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829381.322158:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476946816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829381.322191:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829381.322196:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:2.0:1787829381.322196:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476946432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829381.322200:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829381.322208:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3748 00000100:00080000:1.0:1787829381.322213:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829381.322216:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829381.322217:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829381.322224:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787829381.322225:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829381.322231:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476946432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829381.322236:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476946944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829381.322255:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829381.322286:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829381.322300:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff9a7890b1f0c0 x1874671476946560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829381.322314:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476946560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829381.322323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829381.322326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829381.322330:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829381.322337:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829381.322364:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829381.322377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff9a7890b1fc00 x1874671476946688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829381.322390:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476946688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829381.322398:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829381.322401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829381.322404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829381.322411:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829381.322438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829381.322451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff9a7890b1f840 x1874671476946816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829381.322464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476946816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829381.322471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829381.322473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829381.322475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829381.322479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829381.322493:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829381.322500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829381/real 1787829381] req@ffff9a7890b1c000 x1874671476946944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829381.322507:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476946944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829381.322514:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829381.322516:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829381.322518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476946944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829382.345962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829383.369966:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829384.393958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829385.417905:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829386.441982:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829386.441990:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829386.441994:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829386.442002:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3753 00010000:00010000:0.0:1787829386.442008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829386.442012:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829386.442019:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829386.442051:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476947072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829386.442072:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829386.442075:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829386.442076:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829386.442079:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3753 00000100:00080000:3.0:1787829386.442081:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829386.442083:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829386.442091:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829386.442101:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476947200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829386.442109:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829386.442114:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476947072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829386.442114:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829386.442118:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829386.442125:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3753 00000100:00080000:3.0:1787829386.442127:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829386.442130:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829386.442133:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829386.442143:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476947328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829386.442149:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829386.442151:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829386.442153:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829386.442156:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3753 00000100:00080000:3.0:1787829386.442165:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829386.442168:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829386.442171:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829386.442179:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476947456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829386.442185:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829386.442187:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829386.442188:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829386.442191:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3753 00000100:00080000:3.0:1787829386.442194:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829386.442198:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829386.442202:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829386.442212:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476947584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829386.442225:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829386.442260:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829386.442286:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829386.442305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829387.465966:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829388.490001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829389.514181:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787829389.514200:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787829389.514254:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787829389.514273:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829389.514314:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829389.514325:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829389.514335:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787829389.514354:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787829389.514383:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829389] req@ffff9a78a6776d00 x1874671476947200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829389.514423:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476947200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829389.514439:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829389.514446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829389.514452:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829389.514461:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829389] req@ffff9a78a67761c0 x1874671476947328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829389.514474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476947328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829389.514490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829389.514494:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829389.514497:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829389.514505:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829389] req@ffff9a78a6774780 x1874671476947456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829389.514518:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476947456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829389.514530:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829389.514534:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829389.514538:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829389.514546:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829389] req@ffff9a78a6775680 x1874671476947072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829389.514560:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476947072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829389.514589:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829389.514598:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829389.514602:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476947072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829389.514649:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829386/real 1787829389] req@ffff9a78a67770c0 x1874671476947584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829389.514664:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476947584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829389.514677:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829389.514688:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829389.514692:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829390.537961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829391.561909:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829391.561917:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829391.561920:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829391.561928:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3759 00000100:00080000:3.0:1787829391.561937:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829391.561943:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829391.561976:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476947712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829391.562008:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829391.562011:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829391.562018:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829391.562021:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3759 00000100:00080000:3.0:1787829391.562024:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829391.562027:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829391.562035:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829391.562046:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476947840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829391.562054:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829391.562057:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829391.562058:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829391.562064:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3759 00000100:00100000:2.0:1787829391.562066:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476947712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829391.562066:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829391.562071:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829391.562078:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829391.562090:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476947968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829391.562099:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829391.562103:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829391.562105:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829391.562110:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3759 00000100:00080000:3.0:1787829391.562124:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829391.562128:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829391.562135:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829391.562136:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829391.562149:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476948096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829391.562157:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829391.562159:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829391.562161:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829391.562164:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3759 00000100:00080000:3.0:1787829391.562167:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829391.562169:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829391.562173:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829391.562185:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476948224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829391.562197:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829391/real 1787829391] req@ffff9a78a6776940 x1874671476947712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829391.562222:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476947712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829391.562236:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829391.562241:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829391.562247:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476947712:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787829391.562259:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829391.562281:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829391.562314:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829391.562329:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829391/real 1787829391] req@ffff9a78a6776580 x1874671476947840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829391.562344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476947840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829391.562353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829391.562357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829391.562360:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829391.562368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829391.562390:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829391.562403:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829391/real 1787829391] req@ffff9a78a6774b40 x1874671476947968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829391.562416:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476947968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829391.562424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829391.562428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829391.562431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476947968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829391.562437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829391.562459:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829391.562472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829391/real 1787829391] req@ffff9a78a67752c0 x1874671476948096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829391.562484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476948096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829391.562494:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829391.562497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829391.562500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829391.562506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829391.562526:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829391.562538:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829391/real 1787829391] req@ffff9a78a6775a40 x1874671476948224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829391.562551:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476948224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829391.562577:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829391.562581:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829391.562584:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829392.586039:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829393.609982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829394.633951:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829395.658038:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829396.681939:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829396.681949:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829396.681954:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829396.681971:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3764 00000100:00080000:1.0:1787829396.681982:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829396.681990:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829396.682007:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829396.682028:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476948352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829396.682055:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829396.682059:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829396.682061:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829396.682066:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3764 00000100:00080000:1.0:1787829396.682070:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829396.682075:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829396.682086:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829396.682102:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476948480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829396.682109:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476948352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829396.682119:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829396.682124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829396.682126:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829396.682132:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3764 00000100:00080000:1.0:1787829396.682136:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829396.682140:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829396.682146:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829396.682161:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476948608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829396.682172:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829396.682178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:2.0:1787829396.682179:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829396.682181:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829396.682188:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3764 00000100:00080000:1.0:1787829396.682194:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829396.682199:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829396.682205:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829396.682218:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476948736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829396.682226:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829396.682229:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829396.682233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829396.682234:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829396/real 1787829396] req@ffff9a7890b1e580 x1874671476948352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829396.682237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3764 00000100:00080000:1.0:1787829396.682242:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829396.682249:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829396.682255:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829396.682264:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476948864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829396.682270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476948352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829396.682283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829396.682286:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829396.682290:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476948352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829396.682313:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829396.682344:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829396.682359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829396/real 1787829396] req@ffff9a7890b1da40 x1874671476948480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829396.682373:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476948480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829396.682388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829396.682392:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829396.682396:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829396.682401:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829396.682418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829396.682427:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829396/real 1787829396] req@ffff9a7890b1d680 x1874671476948608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829396.682439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476948608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829396.682448:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829396.682452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829396.682455:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829396.682462:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829396.682483:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829396.682495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829396/real 1787829396] req@ffff9a7890b1de00 x1874671476948736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829396.682503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476948736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829396.682510:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829396.682512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829396.682515:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829396.682534:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829396.682550:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829396.682558:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829396/real 1787829396] req@ffff9a7890b1e940 x1874671476948864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829396.682565:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476948864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829396.682572:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829396.682574:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829396.682576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476948864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829397.705917:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829398.729983:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829399.753999:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829400.777920:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829401.801921:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829401.801930:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829401.801933:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829401.801942:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3769 00000100:00080000:3.0:1787829401.801950:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829401.801956:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829401.801988:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476948992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829401.802009:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829401.802012:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829401.802013:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829401.802016:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3769 00000100:00080000:3.0:1787829401.802018:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829401.802021:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829401.802028:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829401.802037:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476949120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829401.802044:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829401.802046:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829401.802047:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829401.802050:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3769 00000100:00080000:3.0:1787829401.802052:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829401.802054:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829401.802057:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829401.802064:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476948992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829401.802068:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476949248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829401.802078:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829401.802080:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829401.802081:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829401.802084:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3769 00000100:00080000:3.0:1787829401.802086:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829401.802088:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829401.802091:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829401.802101:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476949376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829401.802107:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829401.802109:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829401.802111:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829401.802113:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3769 00000100:00080000:3.0:1787829401.802122:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829401.802125:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829401.802127:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829401.802134:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829401.802137:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476949504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829401.802191:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829401/real 1787829401] req@ffff9a78a6774000 x1874671476948992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829401.802213:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476948992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787829401.802224:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829401.802225:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829401.802232:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829401.802237:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476948992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829401.802247:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829401.802270:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829401.802285:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829401/real 1787829401] req@ffff9a78a6777480 x1874671476949120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829401.802298:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476949120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829401.802307:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829401.802310:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829401.802314:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829401.802337:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829401.802368:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829401.802382:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829401/real 1787829401] req@ffff9a78a6775a40 x1874671476949248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829401.802395:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476949248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829401.802404:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829401.802407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829401.802410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829401.802417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829401.802438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829401.802450:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829401/real 1787829401] req@ffff9a78a67752c0 x1874671476949376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829401.802463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476949376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829401.802473:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829401.802477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829401.802480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829401.802504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829401.802526:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829401.802539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829401/real 1787829401] req@ffff9a78a6774b40 x1874671476949504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829401.802550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476949504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829401.802560:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829401.802564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829401.802567:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829402.825961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829403.849982:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829404.873985:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829405.897984:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829406.921954:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829406.921965:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829406.921971:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787829406.921981:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829406.921982:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3774 00000100:00080000:0.0:1787829406.921995:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829406.922002:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829406.922043:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476949632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829406.922069:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829406.922072:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829406.922074:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829406.922077:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3774 00000100:00080000:0.0:1787829406.922080:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829406.922083:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829406.922092:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829406.922108:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476949760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829406.922123:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476949632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829406.922129:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829406.922134:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829406.922137:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829406.922142:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3774 00000100:00080000:0.0:1787829406.922147:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829406.922151:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829406.922157:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829406.922171:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476949888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829406.922180:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829406.922188:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829406.922193:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829406.922195:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829406.922200:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3774 00000100:00080000:0.0:1787829406.922205:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829406.922209:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829406.922214:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829406.922229:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476950016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829406.922236:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829406/real 1787829406] req@ffff9a7890a3c000 x1874671476949632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829406.922244:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829406.922252:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829406.922257:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829406.922263:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3774 00000100:00100000:2.0:1787829406.922267:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476949632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829406.922269:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829406.922279:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:2.0:1787829406.922281:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787829406.922285:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787829406.922287:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829406.922294:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476949632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829406.922305:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476950144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829406.922317:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829406.922337:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829406.922346:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829406/real 1787829406] req@ffff9a7890a3da40 x1874671476949760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829406.922354:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476949760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829406.922360:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829406.922362:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829406.922365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829406.922368:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829406.922378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829406.922383:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829406/real 1787829406] req@ffff9a7890a3c3c0 x1874671476949888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829406.922390:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476949888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829406.922396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829406.922398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829406.922400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476949888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829406.922403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829406.922412:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829406.922418:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829406/real 1787829406] req@ffff9a7890a3fc00 x1874671476950016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829406.922426:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476950016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829406.922432:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829406.922434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829406.922436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829406.922445:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829406.922476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829406.922484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829406/real 1787829406] req@ffff9a7890a3de00 x1874671476950144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829406.922492:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476950144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829431 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829406.922499:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829406.922501:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829406.922503:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829407.945969:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829408.969974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829409.993958:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829411.017970:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829412.042039:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829412.042076:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829412.042081:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829412.042085:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829412.042092:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3779 00000100:00080000:3.0:1787829412.042100:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829412.042106:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829412.042148:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476950272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829412.042168:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829412.042171:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829412.042173:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829412.042176:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3779 00000100:00080000:3.0:1787829412.042178:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829412.042180:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829412.042188:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829412.042197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476950400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829412.042205:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829412.042208:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476950272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829412.042209:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829412.042212:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829412.042221:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3779 00000100:00080000:3.0:1787829412.042222:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829412.042225:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829412.042228:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829412.042238:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476950528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829412.042245:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829412.042248:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829412.042249:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829412.042252:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3779 00000100:00080000:3.0:1787829412.042264:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829412.042268:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829412.042273:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829412.042274:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829412.042292:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476950656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829412.042304:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829412.042307:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829412.042310:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829412.042315:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3779 00000100:00080000:3.0:1787829412.042319:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829412.042324:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9a78a6776580 x1874671476950272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829412.042325:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829412.042333:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829412.042348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476950272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829412.042357:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1787829412.042359:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476950784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829412.042363:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829412.042370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476950272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829412.042396:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829412.042429:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829412.042440:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9a78a6776940 x1874671476950400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829412.042449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476950400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829412.042455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829412.042458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829412.042460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829412.042464:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829412.042478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829412.042484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9a78a67770c0 x1874671476950528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829412.042491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476950528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829412.042496:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829412.042498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829412.042501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829412.042505:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829412.042515:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829412.042521:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9a78a6775680 x1874671476950656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829412.042528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476950656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829412.042534:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829412.042536:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829412.042538:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829412.042541:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829412.042551:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829412.042556:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829412/real 1787829412] req@ffff9a78a6774780 x1874671476950784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829412.042563:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476950784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829412.042569:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829412.042571:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829412.042573:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476950784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829413.065939:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829414.090002:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829415.114012:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829416.137920:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829417.161933:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829417.161946:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829417.161952:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829417.161966:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3784 00000100:00080000:0.0:1787829417.161980:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829417.161991:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829417.161991:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829417.162048:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476950912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829417.162080:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829417.162085:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829417.162088:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829417.162094:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3784 00000100:00080000:0.0:1787829417.162098:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829417.162103:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829417.162112:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476950912:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787829417.162118:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829417.162141:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476951040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829417.162154:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829417.162158:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829417.162161:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829417.162166:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3784 00000100:00080000:0.0:1787829417.162171:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829417.162178:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787829417.162179:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787829417.162190:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829417.162209:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476951168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829417.162221:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829417.162225:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829417.162227:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829417.162234:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3784 00000100:00100000:2.0:1787829417.162236:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9a7890a3f480 x1874671476950912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829417.162239:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829417.162253:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829417.162269:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829417.162276:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476950912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829417.162282:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476951296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829417.162319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829417.162320:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829417.162328:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829417.162329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787829417.162338:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829417.162345:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3784 00000100:00080000:0.0:1787829417.162347:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829417.162348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476950912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829417.162354:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829417.162361:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829417.162372:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476951424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829417.162394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829417.162423:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829417.162435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9a7890a3de00 x1874671476951040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829417.162449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476951040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829417.162458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829417.162462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829417.162466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829417.162473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829417.162492:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829417.162500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9a7890a3fc00 x1874671476951168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829417.162512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476951168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829417.162518:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829417.162520:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829417.162522:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829417.162528:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829417.162547:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829417.162559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9a7890a3c3c0 x1874671476951296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829417.162572:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476951296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829417.162582:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829417.162584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829417.162586:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829417.162599:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829417.162614:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829417.162621:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829417/real 1787829417] req@ffff9a7890a3da40 x1874671476951424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829417.162628:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476951424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829417.162639:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829417.162641:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829417.162643:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829418.185932:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829419.209928:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829420.233985:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829421.257955:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829422.281933:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829422.281943:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829422.281948:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787829422.281951:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829422.281961:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3789 00000100:00080000:0.0:1787829422.281975:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829422.281984:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829422.282028:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476951552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829422.282051:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829422.282054:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829422.282055:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829422.282058:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3789 00000100:00080000:0.0:1787829422.282061:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829422.282064:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829422.282074:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829422.282082:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476951552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829422.282088:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476951680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829422.282098:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829422.282100:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829422.282102:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829422.282105:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3789 00000100:00080000:0.0:1787829422.282107:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829422.282109:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829422.282112:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829422.282122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476951808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829422.282129:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829422.282132:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829422.282133:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829422.282136:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3789 00000100:00080000:0.0:1787829422.282145:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829422.282148:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829422.282152:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829422.282157:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829422.282164:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476951936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829422.282172:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829422.282175:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829422.282176:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829422.282181:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3789 00000100:00080000:0.0:1787829422.282183:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829422.282186:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829422.282189:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829422.282197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476952064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829422.282222:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff9a7890a3c000 x1874671476951552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829422.282240:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476951552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829422.282249:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829422.282252:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829422.282255:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476951552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829422.282279:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829422.282297:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829422.282306:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff9a7890a3da40 x1874671476951680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829422.282315:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476951680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829422.282321:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829422.282331:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829422.282334:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829422.282339:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829422.282368:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829422.282376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff9a7890a3c3c0 x1874671476951808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829422.282384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476951808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829422.282390:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829422.282393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829422.282395:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829422.282399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829422.282411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829422.282416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff9a7890a3fc00 x1874671476951936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829422.282424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476951936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829422.282431:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829422.282433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829422.282435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476951936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829422.282438:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829422.282448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829422.282454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829422/real 1787829422] req@ffff9a7890a3de00 x1874671476952064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829422.282461:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476952064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829422.282468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829422.282470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829422.282472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829423.306027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829424.330015:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829425.353967:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829426.378003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829427.401957:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.401969:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829427.401975:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.401989:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3794 00000100:00080000:1.0:1787829427.402003:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.402012:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829427.402020:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829427.402063:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476952192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829427.402087:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.402091:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829427.402093:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.402098:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3794 00000100:00080000:1.0:1787829427.402102:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.402107:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829427.402119:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829427.402119:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476952192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829427.402132:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476952320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829427.402148:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.402152:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829427.402154:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.402158:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3794 00000100:00080000:1.0:1787829427.402179:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.402184:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829427.402188:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829427.402192:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829427.402203:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476952448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829427.402212:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.402215:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829427.402217:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.402221:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3794 00000100:00080000:1.0:1787829427.402224:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.402228:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829427.402234:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829427.402244:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476952576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829427.402246:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9a7890b1e940 x1874671476952192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829427.402254:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829427.402257:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829427.402259:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829427.402263:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3794 00000100:00100000:2.0:1787829427.402265:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476952192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829427.402266:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829427.402271:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829427.402275:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787829427.402279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829427.402283:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787829427.402284:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476952704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829427.402288:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476952192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829427.402331:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829427.402354:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829427.402365:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9a7890b1de00 x1874671476952320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829427.402379:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476952320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829427.402385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829427.402387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829427.402390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829427.402394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829427.402406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829427.402412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9a7890b1d680 x1874671476952448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829427.402420:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476952448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829427.402425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829427.402427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829427.402429:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829427.402433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829427.402443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829427.402449:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9a7890b1da40 x1874671476952576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829427.402456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476952576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829427.402468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829427.402470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829427.402472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829427.402475:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829427.402485:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829427.402490:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829427/real 1787829427] req@ffff9a7890b1e580 x1874671476952704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829427.402498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476952704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829427.402505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829427.402507:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829427.402508:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829428.425934:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829429.450032:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829430.473958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829431.498027:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829432.521939:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829432.521955:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829432.521961:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829432.521975:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3800 00000100:00080000:0.0:1787829432.521989:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787829432.521992:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829432.522001:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829432.522044:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476952832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829432.522073:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829432.522077:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829432.522078:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829432.522081:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3800 00000100:00080000:0.0:1787829432.522084:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829432.522087:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829432.522097:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829432.522107:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476952960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829432.522121:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829432.522125:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829432.522126:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476952832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829432.522127:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829432.522137:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3800 00000100:00080000:0.0:1787829432.522139:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829432.522149:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829432.522155:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829432.522165:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476953088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829432.522172:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829432.522176:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829432.522177:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829432.522180:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3800 00000100:00080000:0.0:1787829432.522191:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829432.522195:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829432.522202:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829432.522202:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829432.522221:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476953216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829432.522232:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829432.522236:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829432.522238:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829432.522244:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3800 00000100:00080000:0.0:1787829432.522250:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829432.522251:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829432/real 1787829432] req@ffff9a7890a3f480 x1874671476952832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829432.522259:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829432.522283:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829432.522291:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476952832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829432.522302:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476953344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829432.522309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829432.522318:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829432.522323:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476952832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829432.522348:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829432.522367:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829432.522377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829432/real 1787829432] req@ffff9a7890a3de00 x1874671476952960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829432.522386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476952960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829432.522391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829432.522394:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829432.522396:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476952960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829432.522400:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829432.522410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829432.522416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829432/real 1787829432] req@ffff9a7890a3fc00 x1874671476953088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829432.522424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476953088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829432.522429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829432.522431:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829432.522434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829432.522438:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829432.522449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829432.522455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829432/real 1787829432] req@ffff9a7890a3c3c0 x1874671476953216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829432.522463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476953216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829432.522470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829432.522471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829432.522473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829432.522477:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829432.522487:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829432.522493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829432/real 1787829432] req@ffff9a7890a3da40 x1874671476953344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829432.522501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476953344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829432.522508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829432.522510:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829432.522511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829433.546067:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829434.570001:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829435.594034:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829436.618019:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829437.641956:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787829437.641956:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829437.641967:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829437.641971:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829437.641982:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3805 00000100:00080000:1.0:1787829437.641992:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829437.641999:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829437.642036:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476953472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829437.642063:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829437.642067:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829437.642069:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829437.642073:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3805 00000100:00080000:1.0:1787829437.642078:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829437.642082:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829437.642092:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829437.642109:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476953600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829437.642121:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829437.642127:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829437.642129:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829437.642134:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3805 00000100:00080000:1.0:1787829437.642137:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829437.642144:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829437.642154:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829437.642154:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476953472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829437.642175:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476953728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829437.642189:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829437.642193:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829437.642195:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829437.642199:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3805 00000100:00080000:1.0:1787829437.642203:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829437.642207:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829437.642213:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829437.642226:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476953856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829437.642248:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829437.642252:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829437.642254:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829437.642257:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3805 00000100:00080000:1.0:1787829437.642262:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787829437.642268:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829437.642269:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829437.642280:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829437.642296:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476953984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829437.642344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829437/real 1787829437] req@ffff9a7890b1c000 x1874671476953472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829437.642369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476953472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829437.642382:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829437.642387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829437.642391:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476953472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829437.642400:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829437.642422:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829437.642432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829437/real 1787829437] req@ffff9a7890b1f840 x1874671476953600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829437.642441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476953600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829437.642447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829437.642450:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829437.642452:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829437.642478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829437.642489:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829437.642496:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829437/real 1787829437] req@ffff9a7890b1fc00 x1874671476953728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829437.642505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476953728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829437.642510:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829437.642512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829437.642514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829437.642518:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829437.642538:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829437.642545:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829437/real 1787829437] req@ffff9a7890b1f0c0 x1874671476953856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829437.642554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476953856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829437.642562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829437.642564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829437.642568:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829437.642575:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829437.642595:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829437.642607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829437/real 1787829437] req@ffff9a7890b1f480 x1874671476953984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829437.642619:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476953984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829437.642637:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829437.642641:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829437.642645:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476953984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829438.666026:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829439.689991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829440.713925:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829441.737980:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829442.761975:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829442.761986:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829442.761991:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829442.762003:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3810 00000100:00080000:2.0:1787829442.762016:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829442.762023:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829442.762033:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829442.762072:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476954112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829442.762104:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829442.762108:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829442.762109:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829442.762113:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3810 00000100:00080000:2.0:1787829442.762116:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829442.762121:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829442.762133:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829442.762147:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476954112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829442.762152:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476954240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829442.762175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829442.762179:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829442.762181:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829442.762187:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3810 00000100:00080000:2.0:1787829442.762192:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829442.762197:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829442.762218:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829442.762234:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476954368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829442.762242:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829442.762248:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829442.762255:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829442.762258:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829442.762264:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3810 00000100:00080000:2.0:1787829442.762269:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829442.762274:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829442.762280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829442.762292:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476954496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829442.762302:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829442.762306:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829442.762308:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829442.762314:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829442/real 1787829442] req@ffff9a78841db480 x1874671476954112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829442.762316:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3810 00000100:00080000:2.0:1787829442.762324:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829442.762333:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829442.762340:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829442.762347:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476954112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829442.762354:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476954624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829442.762372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829442.762380:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829442.762386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476954112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829442.762416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829442.762440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829442.762453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829442/real 1787829442] req@ffff9a78841d83c0 x1874671476954240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829442.762467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476954240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829442.762477:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829442.762480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829442.762483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829442.762490:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829442.762510:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829442.762522:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829442/real 1787829442] req@ffff9a78841d8000 x1874671476954368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829442.762535:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476954368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829442.762544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829442.762547:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829442.762551:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829442.762558:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829442.762581:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829442.762593:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829442/real 1787829442] req@ffff9a78841da580 x1874671476954496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829442.762606:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476954496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829442.762617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829442.762620:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829442.762623:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829442.762629:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829442.762649:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829442.762660:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829442/real 1787829442] req@ffff9a78841d9e00 x1874671476954624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829442.762673:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476954624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829442.762684:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829442.762687:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829442.762691:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787829443.785956:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829444.809984:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829445.833980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829446.857961:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829447.881975:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829447.881979:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829447.881988:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829447.881992:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829447.882003:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3815 00000100:00080000:1.0:1787829447.882012:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829447.882017:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829447.882048:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476954752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829447.882069:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829447.882071:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829447.882073:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829447.882076:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3815 00000100:00080000:1.0:1787829447.882078:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829447.882082:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829447.882089:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829447.882102:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476954880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829447.882102:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476954752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829447.882116:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829447.882118:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829447.882119:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829447.882123:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3815 00000100:00080000:1.0:1787829447.882126:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829447.882129:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829447.882132:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829447.882148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476955008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829447.882158:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829447.882161:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787829447.882175:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829447.882176:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829447.882184:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3815 00000100:00080000:1.0:1787829447.882188:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829447.882191:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829447.882197:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829447.882207:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476955136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829447.882214:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829447.882216:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829447.882218:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829447.882221:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3815 00000100:00080000:1.0:1787829447.882224:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829447.882228:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829447/real 1787829447] req@ffff9a7890b1f480 x1874671476954752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829447.882230:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829447.882237:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829447.882253:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476954752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829447.882255:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476955264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829447.882269:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829447.882275:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829447.882281:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476954752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829447.882307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829447.882327:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829447.882336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829447/real 1787829447] req@ffff9a7890b1f0c0 x1874671476954880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829447.882344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476954880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829447.882350:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829447.882352:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829447.882355:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476954880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829447.882359:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829447.882369:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829447.882375:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829447/real 1787829447] req@ffff9a7890b1fc00 x1874671476955008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829447.882388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476955008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829447.882396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829447.882400:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829447.882403:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829447.882409:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829447.882430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829447.882443:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829447/real 1787829447] req@ffff9a7890b1f840 x1874671476955136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829447.882453:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476955136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829447.882460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829447.882463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829447.882465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829447.882469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829447.882482:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829447.882489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829447/real 1787829447] req@ffff9a7890b1c000 x1874671476955264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829447.882496:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476955264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829447.882506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829447.882508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829447.882511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829448.906023:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829449.929929:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829450.954024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829451.978054:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829453.001937:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829453.001945:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829453.001949:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829453.001957:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3820 00010000:00010000:3.0:1787829453.001959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829453.001967:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829453.001974:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829453.002005:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476955392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829453.002023:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829453.002026:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829453.002027:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829453.002030:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3820 00000100:00080000:1.0:1787829453.002032:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829453.002035:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829453.002043:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829453.002055:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476955520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829453.002056:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476955392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829453.002071:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829453.002073:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829453.002074:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829453.002077:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3820 00000100:00080000:1.0:1787829453.002078:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829453.002080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829453.002085:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829453.002093:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476955648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829453.002100:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829453.002103:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829453.002104:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829453.002106:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3820 00000100:00080000:1.0:1787829453.002116:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829453.002118:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829453.002121:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829453.002142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476955776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829453.002149:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829453.002152:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829453.002153:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829453.002157:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3820 00000100:00080000:1.0:1787829453.002161:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829453.002164:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829453.002169:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829453.002177:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476955904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829453.002185:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829453.002220:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829453.002243:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829453.002268:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829454.025967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829455.049962:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829456.073991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829456.074077:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787829456.074132:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829456.074147:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829456.074184:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829456.074193:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829456.074206:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829456.074218:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829456.074234:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829456] req@ffff9a7890b1da40 x1874671476955520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829456.074268:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476955520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829456.074279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829456.074285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829456.074290:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829456.074297:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829456] req@ffff9a7890b1d680 x1874671476955648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829456.074317:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476955648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829456.074325:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829456.074328:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829456.074331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829456.074337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829456] req@ffff9a7890b1de00 x1874671476955776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829456.074348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476955776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829456.074358:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829456.074361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829456.074364:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829456.074370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829456] req@ffff9a7890b1e580 x1874671476955392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829456.074381:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476955392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829456.074388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829456.074391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829456.074394:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476955392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829456.074421:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829453/real 1787829456] req@ffff9a7890b1e940 x1874671476955904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829456.074433:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476955904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829456.074441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829456.074446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829456.074449:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476955904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829457.098093:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829458.121973:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829458.121984:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829458.121988:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829458.121999:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3825 00000100:00080000:0.0:1787829458.122010:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829458.122018:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829458.122065:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476956032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787829458.122078:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829458.122115:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829458.122121:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829458.122124:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829458.122128:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3825 00000100:00080000:0.0:1787829458.122133:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829458.122138:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476956032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829458.122141:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829458.122159:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829458.122176:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476956160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829458.122188:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829458.122192:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829458.122194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829458.122199:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3825 00000100:00080000:0.0:1787829458.122212:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829458.122220:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829458.122222:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787829458.122227:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829458.122246:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476956288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829458.122254:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829458.122257:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829458.122258:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829458.122262:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3825 00000100:00080000:0.0:1787829458.122265:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829458.122268:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829458.122273:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829458.122285:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476956416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829458.122285:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9a7890a3c000 x1874671476956032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829458.122316:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829458.122320:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829458.122323:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829458.122331:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3825 00000100:00100000:3.0:1787829458.122331:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476956032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829458.122339:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829458.122348:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829458.122356:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787829458.122357:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829458.122368:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829458.122376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476956032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829458.122378:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476956544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829458.122415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829458.122453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829458.122470:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9a7890a3da40 x1874671476956160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829458.122483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476956160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829458.122499:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829458.122503:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829458.122507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829458.122513:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829458.122537:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829458.122550:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9a7890a3c3c0 x1874671476956288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829458.122575:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476956288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829458.122585:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829458.122589:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829458.122592:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829458.122598:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829458.122621:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829458.122634:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9a7890a3fc00 x1874671476956416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829458.122647:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476956416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829458.122658:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829458.122662:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829458.122667:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829458.122674:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829458.122694:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829458.122708:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829458/real 1787829458] req@ffff9a7890a3de00 x1874671476956544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829458.122719:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476956544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829458.122729:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829458.122732:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829458.122735:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829459.146136:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829460.170037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829461.193945:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829462.218039:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829463.241930:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829463.241941:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829463.241945:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829463.241957:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3830 00000100:00080000:0.0:1787829463.241969:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829463.241977:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829463.242011:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829463.242015:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476956672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829463.242036:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829463.242039:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829463.242041:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829463.242044:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3830 00000100:00080000:0.0:1787829463.242046:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829463.242050:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829463.242069:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829463.242090:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476956800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829463.242091:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476956672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829463.242110:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829463.242113:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829463.242116:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829463.242121:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3830 00000100:00080000:0.0:1787829463.242125:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829463.242130:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829463.242136:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829463.242151:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476956928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829463.242163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787829463.242167:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829463.242169:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829463.242173:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829463.242180:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3830 00000100:00080000:0.0:1787829463.242185:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829463.242190:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829463.242197:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829463.242211:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476957056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829463.242222:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829463.242225:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787829463.242225:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9a7890a3f480 x1874671476956672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829463.242229:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829463.242238:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3830 00000100:00080000:0.0:1787829463.242243:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829463.242247:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829463.242251:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476956672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787829463.242253:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829463.242267:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787829463.242271:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476957184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829463.242274:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829463.242281:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476956672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829463.242306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829463.242324:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829463.242336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9a7890a3de00 x1874671476956800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829463.242349:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476956800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829463.242358:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829463.242361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829463.242365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829463.242371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829463.242391:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829463.242403:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9a7890a3fc00 x1874671476956928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829463.242414:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476956928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829463.242422:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829463.242425:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829463.242428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476956928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829463.242434:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829463.242451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829463.242462:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9a7890a3c3c0 x1874671476957056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829463.242474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476957056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829463.242484:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829463.242488:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829463.242507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829463.242512:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829463.242527:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829463.242535:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829463/real 1787829463] req@ffff9a7890a3da40 x1874671476957184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829463.242543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476957184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829463.242549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829463.242551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829463.242553:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829464.265995:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829465.289962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829466.314003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829467.337966:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829468.361946:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829468.361955:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829468.361958:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829468.361967:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3835 00000100:00080000:2.0:1787829468.361977:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829468.361982:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829468.361984:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829468.362018:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476957312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829468.362051:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829468.362053:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829468.362055:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829468.362058:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3835 00000100:00080000:2.0:1787829468.362060:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829468.362063:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829468.362072:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829468.362081:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476957440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829468.362089:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829468.362092:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829468.362095:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829468.362096:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476957312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829468.362100:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3835 00000100:00080000:2.0:1787829468.362105:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829468.362108:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829468.362111:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829468.362120:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476957568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829468.362126:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829468.362129:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829468.362131:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829468.362134:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3835 00000100:00080000:2.0:1787829468.362152:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829468.362154:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829468.362158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829468.362166:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476957696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829468.362171:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829468.362178:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829468.362181:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829468.362182:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829468.362186:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3835 00000100:00080000:2.0:1787829468.362188:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829468.362191:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829468.362194:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829468.362200:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476957824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829468.362229:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff9a78841d8780 x1874671476957312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.362248:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476957312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.362260:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.362264:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829468.362269:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476957312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829468.362298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829468.362319:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829468.362331:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff9a78841d9a40 x1874671476957440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.362344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476957440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.362353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.362356:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829468.362360:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829468.362366:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829468.362382:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829468.362392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff9a78841da1c0 x1874671476957568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.362404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476957568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.362413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.362417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829468.362420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829468.362427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829468.362443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829468.362461:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff9a78841db0c0 x1874671476957696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.362474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476957696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.362484:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.362487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829468.362491:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829468.362497:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829468.362512:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829468.362522:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829468/real 1787829468] req@ffff9a78841dbc00 x1874671476957824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829468.362540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476957824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829468.362551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829468.362555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829468.362558:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476957824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829469.386043:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829470.409964:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829471.433978:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829472.457972:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829473.481929:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829473.481937:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829473.481941:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829473.481949:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3840 00000100:00080000:1.0:1787829473.481959:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829473.481965:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829473.481965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829473.482012:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476957952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829473.482031:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829473.482034:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829473.482035:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829473.482038:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3840 00000100:00080000:1.0:1787829473.482040:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829473.482043:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829473.482051:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829473.482060:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476958080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829473.482068:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829473.482070:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829473.482071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829473.482077:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3840 00000100:00080000:1.0:1787829473.482080:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829473.482080:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476957952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829473.482084:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829473.482091:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829473.482098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476958208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829473.482110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829473.482112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829473.482113:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829473.482117:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3840 00000100:00080000:1.0:1787829473.482119:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829473.482121:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829473.482124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829473.482132:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476958336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829473.482142:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829473.482146:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829473.482148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829473.482152:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3840 00000100:00080000:1.0:1787829473.482156:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829473.482159:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829473.482163:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829473.482170:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829473.482183:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476958464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829473.482208:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829473/real 1787829473] req@ffff9a7890b1e940 x1874671476957952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829473.482223:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476957952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829473.482232:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829473.482235:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829473.482244:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476957952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829473.482263:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829473.482279:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829473.482288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829473/real 1787829473] req@ffff9a7890b1e580 x1874671476958080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829473.482296:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476958080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829473.482302:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829473.482304:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829473.482306:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829473.482310:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829473.482319:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829473.482325:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829473/real 1787829473] req@ffff9a7890b1de00 x1874671476958208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829473.482332:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476958208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829473.482338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829473.482339:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829473.482342:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829473.482346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829473.482355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829473.482363:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829473/real 1787829473] req@ffff9a7890b1d680 x1874671476958336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829473.482375:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476958336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829473.482385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829473.482389:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829473.482392:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829473.482398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829473.482419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829473.482431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829473/real 1787829473] req@ffff9a7890b1da40 x1874671476958464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829473.482455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476958464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829473.482466:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829473.482470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829473.482474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829474.505964:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829475.530000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829476.554008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829477.578053:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829478.601970:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787829478.601972:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829478.601982:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829478.601986:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829478.601996:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3846 00000100:00080000:0.0:1787829478.602007:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829478.602014:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829478.602055:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476958592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829478.602083:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829478.602086:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829478.602087:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829478.602090:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3846 00000100:00080000:0.0:1787829478.602093:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829478.602097:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829478.602107:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829478.602120:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476958720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829478.602126:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476958592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829478.602132:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829478.602141:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829478.602145:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829478.602151:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3846 00000100:00080000:0.0:1787829478.602154:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829478.602159:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829478.602165:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829478.602180:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476958848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829478.602191:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829478.602196:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829478.602198:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829478.602203:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3846 00000800:00000100:3.0:1787829478.602208:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829478.602213:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829478.602221:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829478.602229:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829478.602250:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476958976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829478.602259:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829478/real 1787829478] req@ffff9a7890a3c000 x1874671476958592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829478.602264:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829478.602271:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829478.602274:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829478.602286:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3846 00000100:00080000:0.0:1787829478.602295:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829478.602299:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829478.602304:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829478.602317:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476958592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829478.602323:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476959104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829478.602329:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829478.602341:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829478.602352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476958592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829478.602381:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829478.602403:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829478.602416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829478/real 1787829478] req@ffff9a7890a3da40 x1874671476958720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829478.602429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476958720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829478.602437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829478.602440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829478.602442:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829478.602446:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829478.602460:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829478.602475:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829478/real 1787829478] req@ffff9a7890a3c3c0 x1874671476958848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829478.602483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476958848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829478.602492:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829478.602494:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829478.602496:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829478.602500:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829478.602511:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829478.602516:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829478/real 1787829478] req@ffff9a7890a3fc00 x1874671476958976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829478.602523:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476958976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829478.602529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829478.602530:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829478.602532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476958976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829478.602535:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829478.602545:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829478.602550:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829478/real 1787829478] req@ffff9a7890a3de00 x1874671476959104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829478.602556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476959104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829478.602561:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829478.602562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829478.602564:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829479.626020:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829480.649984:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829481.673992:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829482.697948:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829483.721907:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829483.721930:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829483.721935:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829483.721945:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3851 00010000:00010000:2.0:1787829483.721955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829483.721956:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829483.721964:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829483.721997:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476959232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829483.722019:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829483.722022:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829483.722024:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829483.722028:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3851 00000100:00080000:1.0:1787829483.722032:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829483.722036:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829483.722045:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829483.722049:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476959232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829483.722060:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476959360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829483.722070:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829483.722074:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829483.722076:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829483.722080:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3851 00000100:00080000:1.0:1787829483.722092:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829483.722096:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829483.722100:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829483.722104:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829483.722118:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476959488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829483.722128:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829483.722131:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829483.722133:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829483.722136:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3851 00000100:00080000:1.0:1787829483.722140:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829483.722143:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829483/real 1787829483] req@ffff9a7890b1c000 x1874671476959232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829483.722147:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829483.722157:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829483.722163:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476959232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829483.722171:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476959616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829483.722172:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829483.722178:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829483.722184:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476959232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829483.722191:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829483.722195:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829483.722197:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829483.722202:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3851 00000100:00080000:1.0:1787829483.722206:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829483.722206:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959360:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829483.722214:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829483.722220:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787829483.722226:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829483.722234:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476959744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829483.722236:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829483/real 1787829483] req@ffff9a7890b1f840 x1874671476959360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829483.722263:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476959360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829483.722271:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829483.722273:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829483.722275:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829483.722284:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829483.722294:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829483.722300:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829483/real 1787829483] req@ffff9a7890b1fc00 x1874671476959488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829483.722307:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476959488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829483.722312:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829483.722314:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829483.722315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829483.722318:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829483.722327:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829483.722335:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829483/real 1787829483] req@ffff9a7890b1f0c0 x1874671476959616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829483.722342:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476959616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829483.722347:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829483.722348:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829483.722350:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829483.722359:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829483.722369:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829483.722374:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829483/real 1787829483] req@ffff9a7890b1f480 x1874671476959744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829483.722380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476959744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829483.722386:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829483.722388:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829483.722390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476959744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829484.745958:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829485.769942:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829486.793963:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829487.817970:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829488.841946:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829488.841960:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829488.841967:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829488.841981:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3856 00000100:00080000:0.0:1787829488.841995:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787829488.842000:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829488.842007:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829488.842056:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476959872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829488.842087:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829488.842092:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829488.842094:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829488.842099:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3856 00000100:00080000:0.0:1787829488.842104:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829488.842110:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829488.842123:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829488.842131:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476959872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829488.842142:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476960000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829488.842157:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829488.842162:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829488.842165:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829488.842170:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3856 00000100:00080000:0.0:1787829488.842174:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829488.842179:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829488.842185:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829488.842207:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476960128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829488.842212:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829488.842225:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829488.842230:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829488.842239:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829488.842246:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3856 00000100:00080000:0.0:1787829488.842251:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829488.842256:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829488.842264:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829488.842276:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829488/real 1787829488] req@ffff9a7890a3f480 x1874671476959872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829488.842280:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476960256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829488.842306:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829488.842311:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829488.842315:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829488.842315:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476959872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829488.842325:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3856 00000100:00080000:0.0:1787829488.842333:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829488.842337:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829488.842340:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829488.842346:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:0.0:1787829488.842350:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829488.842353:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476959872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829488.842368:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476960384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829488.842387:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829488.842415:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829488.842429:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829488/real 1787829488] req@ffff9a7890a3de00 x1874671476960000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829488.842443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476960000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829488.842453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829488.842458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829488.842462:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829488.842469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829488.842488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829488.842499:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829488/real 1787829488] req@ffff9a7890a3fc00 x1874671476960128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829488.842513:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476960128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829488.842522:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829488.842525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829488.842529:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829488.842536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829488.842555:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829488.842566:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829488/real 1787829488] req@ffff9a7890a3c3c0 x1874671476960256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829488.842579:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476960256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829488.842591:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829488.842594:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829488.842598:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829488.842604:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829488.842622:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829488.842634:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829488/real 1787829488] req@ffff9a7890a3da40 x1874671476960384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829488.842647:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476960384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829488.842658:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829488.842662:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829488.842665:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829489.866085:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829490.890059:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829491.914006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829492.938017:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829493.961969:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829493.961977:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829493.961980:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829493.961988:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3861 00000100:00080000:1.0:1787829493.961996:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829493.962001:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829493.962013:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829493.962033:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476960512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829493.962051:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829493.962054:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829493.962055:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829493.962057:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3861 00000100:00080000:1.0:1787829493.962060:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829493.962065:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829493.962073:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829493.962079:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476960512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829493.962089:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476960640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829493.962101:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829493.962104:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829493.962107:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829493.962110:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3861 00000100:00080000:1.0:1787829493.962125:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829493.962128:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829493.962133:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829493.962134:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829493.962148:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476960768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829493.962157:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829493.962160:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829493.962162:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829493.962166:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3861 00000100:00080000:1.0:1787829493.962170:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829493.962173:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829493.962178:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829493.962188:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476960896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829493.962197:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829493.962200:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829493/real 1787829493] req@ffff9a7890b1f480 x1874671476960512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829493.962201:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829493.962204:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829493.962211:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3861 00000100:00080000:1.0:1787829493.962216:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829493.962220:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829493.962225:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829493.962226:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476960512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829493.962239:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476961024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829493.962239:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829493.962248:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829493.962254:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476960512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829493.962278:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829493.962299:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829493.962311:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829493/real 1787829493] req@ffff9a7890b1f0c0 x1874671476960640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829493.962324:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476960640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829493.962339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829493.962342:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829493.962346:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829493.962352:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829493.962370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829493.962387:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829493/real 1787829493] req@ffff9a7890b1fc00 x1874671476960768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829493.962398:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476960768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829493.962407:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829493.962410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829493.962412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829493.962417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829493.962434:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829493.962445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829493/real 1787829493] req@ffff9a7890b1f840 x1874671476960896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829493.962456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476960896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829493.962464:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829493.962467:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829493.962470:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476960896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829493.962476:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829493.962494:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829493.962504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829493/real 1787829493] req@ffff9a7890b1c000 x1874671476961024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829493.962515:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476961024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829518 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829493.962525:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829493.962527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829493.962529:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829494.986033:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829496.010017:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829497.034017:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829498.057990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829499.081946:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.081956:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829499.081960:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829499.081971:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3866 00000100:00080000:1.0:1787829499.081982:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829499.081991:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829499.082005:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829499.082027:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476961152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829499.082053:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.082066:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829499.082068:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829499.082073:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3866 00000100:00080000:1.0:1787829499.082077:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829499.082082:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829499.082092:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829499.082094:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476961152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829499.082108:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476961280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829499.082120:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.082124:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829499.082126:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829499.082130:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3866 00000100:00080000:1.0:1787829499.082135:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829499.082140:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829499.082143:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829499.082146:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829499.082162:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476961408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829499.082172:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.082175:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829499.082177:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829499.082182:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9a7890b1da40 x1874671476961152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829499.082183:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3866 00000100:00080000:1.0:1787829499.082189:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829499.082196:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829499.082199:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476961152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829499.082207:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787829499.082210:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787829499.082210:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829499.082213:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476961152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829499.082228:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476961536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829499.082233:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961280:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829499.082246:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829499.082250:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000800:00000100:3.0:1787829499.082251:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829499.082253:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829499.082260:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3866 00000100:00100000:3.0:1787829499.082263:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9a7890b1d680 x1874671476961280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829499.082266:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829499.082274:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476961280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829499.082275:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829499.082286:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829499.082286:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829499.082292:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829499.082297:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829499.082302:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476961664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829499.082305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829499.082340:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829499.082349:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9a7890b1de00 x1874671476961408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829499.082356:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476961408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829499.082361:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829499.082363:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829499.082365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829499.082374:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829499.082384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829499.082390:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9a7890b1e580 x1874671476961536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829499.082396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476961536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829499.082402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829499.082403:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829499.082405:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829499.082408:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829499.082416:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829499.082421:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829499/real 1787829499] req@ffff9a7890b1e940 x1874671476961664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829499.082427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476961664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829499.082432:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829499.082434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829499.082435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829500.106006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829501.130053:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829502.153950:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829503.177971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829504.201956:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829504.201966:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829504.201970:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829504.201980:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3871 00010000:00010000:0.0:1787829504.201981:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829504.201992:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829504.201999:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829504.202029:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476961792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829504.202051:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829504.202055:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829504.202057:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829504.202060:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3871 00000100:00080000:3.0:1787829504.202064:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829504.202068:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829504.202076:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829504.202090:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476961792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829504.202090:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476961920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829504.202106:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829504.202109:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829504.202110:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829504.202114:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3871 00000100:00080000:3.0:1787829504.202118:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829504.202121:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829504.202126:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829504.202135:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476962048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829504.202143:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829504.202146:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829504.202148:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829504.202151:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3871 00000800:00000100:2.0:1787829504.202156:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787829504.202157:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829504.202163:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829504.202175:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829504.202186:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476962176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829504.202193:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829504.202196:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829504.202198:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829504.202202:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3871 00000100:00080000:3.0:1787829504.202205:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829504.202209:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829504.202213:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829504.202217:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9a78a67761c0 x1874671476961792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829504.202228:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476962304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829504.202245:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476961792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.202256:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.202261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829504.202264:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476961792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829504.202286:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829504.202308:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829504.202321:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9a78a6776d00 x1874671476961920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829504.202333:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476961920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.202341:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.202344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829504.202347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476961920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829504.202353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829504.202373:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829504.202384:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9a78a6775e00 x1874671476962048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829504.202395:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476962048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.202402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.202405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829504.202408:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829504.202414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829504.202433:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829504.202444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9a78a6777840 x1874671476962176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829504.202454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476962176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.202462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.202465:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829504.202468:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829504.202473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829504.202490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829504.202500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9a78a6774f00 x1874671476962304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829504.202511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476962304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.202519:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.202522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829504.202526:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829505.226002:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829506.249952:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829507.273981:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829508.297952:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829509.321963:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829509.321976:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829509.321982:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829509.321996:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3876 00000100:00080000:2.0:1787829509.322008:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829509.322016:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829509.322036:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829509.322069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476962432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829509.322095:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829509.322099:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829509.322101:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829509.322106:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3876 00000100:00080000:2.0:1787829509.322111:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829509.322116:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829509.322128:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829509.322131:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476962432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829509.322145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476962560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829509.322156:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829509.322158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829509.322160:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829509.322163:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3876 00000100:00080000:2.0:1787829509.322166:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829509.322169:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829509.322173:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829509.322182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476962688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829509.322189:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829509.322191:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829509.322193:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829509.322196:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3876 00000100:00080000:2.0:1787829509.322199:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829509.322202:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829509.322206:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829509.322210:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829509.322219:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476962816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829509.322227:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829509.322229:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829509.322231:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829509.322234:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3876 00000100:00080000:2.0:1787829509.322236:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829509.322238:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829509.322241:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829509.322249:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476962944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829509.322273:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9a78841dbc00 x1874671476962432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.322291:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476962432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.322300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.322304:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829509.322307:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476962432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829509.322329:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829509.322345:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829509.322354:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9a78841db0c0 x1874671476962560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.322362:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476962560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.322368:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.322370:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829509.322372:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829509.322376:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829509.322385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829509.322391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9a78841da1c0 x1874671476962688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.322398:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476962688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.322403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.322405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829509.322407:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829509.322410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829509.322422:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829509.322428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9a78841d9a40 x1874671476962816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.322440:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476962816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.322446:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.322448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829509.322450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829509.322454:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829509.322463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829509.322469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829509] req@ffff9a78841d8780 x1874671476962944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829509.322476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476962944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829509.322483:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829509.322484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829509.322486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476962944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829510.346054:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829511.370026:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829512.394049:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829513.417982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829514.441946:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829514.441958:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829514.441964:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829514.441977:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3881 00010000:00010000:0.0:1787829514.441991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829514.441991:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829514.442002:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829514.442046:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476963072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829514.442076:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829514.442080:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829514.442083:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829514.442090:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3881 00000100:00080000:2.0:1787829514.442100:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829514.442106:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829514.442114:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476963072:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829514.442122:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829514.442143:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476963200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829514.442156:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829514.442160:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829514.442162:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829514.442167:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3881 00000100:00080000:2.0:1787829514.442179:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829514.442183:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829514.442189:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829514.442192:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829514.442209:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476963328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829514.442219:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829514.442223:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829514.442225:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829514.442230:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3881 00000100:00080000:2.0:1787829514.442235:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829514.442240:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829514.442247:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829514.442254:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9a78841d9e00 x1874671476963072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829514.442264:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476963456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829514.442282:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829514.442287:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476963072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829514.442289:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829514.442293:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829514.442303:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3881 00000100:00080000:2.0:1787829514.442312:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829514.442323:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829514.442336:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829514.442341:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.442346:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829514.442354:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476963584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829514.442356:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476963072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829514.442393:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829514.442413:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829514.442422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9a78841da580 x1874671476963200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829514.442429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476963200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829514.442434:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.442436:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829514.442445:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829514.442449:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829514.442459:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829514.442465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9a78841d8000 x1874671476963328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829514.442471:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476963328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829514.442476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.442477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829514.442480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829514.442483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829514.442492:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829514.442497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9a78841d83c0 x1874671476963456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829514.442503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476963456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829514.442509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.442510:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829514.442512:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829514.442515:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829514.442523:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829514.442528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9a78841db480 x1874671476963584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829514.442534:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476963584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829514.442539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829514.442541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829514.442542:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829515.466028:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829516.490065:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829517.513991:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829518.538025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829519.561943:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829519.561956:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829519.561964:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829519.561970:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829519.561982:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3887 00000100:00080000:0.0:1787829519.561995:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829519.562004:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829519.562050:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476963712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829519.562081:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829519.562087:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829519.562089:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829519.562092:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3887 00000100:00080000:0.0:1787829519.562098:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829519.562104:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829519.562116:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829519.562130:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476963712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829519.562135:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476963840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829519.562152:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829519.562156:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829519.562159:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829519.562164:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3887 00000100:00080000:0.0:1787829519.562168:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829519.562174:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829519.562181:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829519.562198:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476963968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829519.562211:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829519.562228:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829519.562230:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829519.562236:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963840:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787829519.562237:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3887 00000100:00080000:0.0:1787829519.562248:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829519.562255:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829519.562262:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829519.562274:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787829519.562279:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476964096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829519.562296:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829519.562301:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829519.562305:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829519.562313:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3887 00000100:00100000:3.0:1787829519.562313:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964096:192.168.203.140@tcp:8:kworker.0 00000100:00080000:0.0:1787829519.562325:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829519.562334:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829519.562342:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829519.562360:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476964224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829519.562422:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829520.586033:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829521.609956:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829522.634018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829522.634074:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787829522.634112:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829522.634127:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829522.634161:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829522.634171:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829522.634179:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829522.634188:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829522.634206:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829522] req@ffff9a7890a3de00 x1874671476964224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829522.634230:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476964224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829522.634243:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829522.634247:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829522.634252:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964224:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829522.634259:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829522] req@ffff9a7890a3fc00 x1874671476964096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829522.634270:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476964096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829522.634280:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829522.634283:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829522.634286:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829522.634292:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829522] req@ffff9a7890a3da40 x1874671476963840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829522.634303:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476963840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829522.634311:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829522.634314:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829522.634317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829522.634323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829522] req@ffff9a7890a3c3c0 x1874671476963968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829522.634333:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476963968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829522.634341:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829522.634344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829522.634347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476963968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829522.634353:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829522] req@ffff9a7890a3c000 x1874671476963712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829522.634365:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476963712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829522.634372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829522.634375:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829522.634378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476963712:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829523.657968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829524.681990:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829524.682000:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829524.682006:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829524.682015:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3892 00000100:00080000:0.0:1787829524.682026:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787829524.682027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829524.682035:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829524.682070:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476964352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829524.682094:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829524.682098:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829524.682100:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829524.682104:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3892 00000100:00080000:0.0:1787829524.682107:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829524.682112:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829524.682121:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829524.682137:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476964480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829524.682138:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476964352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829524.682155:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829524.682159:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829524.682161:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829524.682165:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3892 00000100:00080000:0.0:1787829524.682169:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829524.682172:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829524.682178:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829524.682189:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476964608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829524.682199:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829524.682204:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829524.682205:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829524.682210:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829524.682215:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3892 00000100:00080000:0.0:1787829524.682221:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829524.682225:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829524.682231:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829524.682243:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476964736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829524.682252:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829524.682255:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829524.682257:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829524.682261:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3892 00000100:00080000:0.0:1787829524.682265:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829524.682269:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829524.682274:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9a7890a3f480 x1874671476964352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787829524.682275:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829524.682292:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476964864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829524.682299:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476964352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829524.682313:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829524.682317:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829524.682322:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476964352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829524.682345:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829524.682367:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829524.682393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9a7890a3c000 x1874671476964480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829524.682406:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476964480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829524.682414:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829524.682417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829524.682420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829524.682426:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829524.682445:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829524.682456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9a7890a3c3c0 x1874671476964608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829524.682468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476964608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829524.682494:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829524.682498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829524.682501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829524.682507:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829524.682525:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829524.682536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9a7890a3da40 x1874671476964736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829524.682547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476964736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829524.682556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829524.682559:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829524.682562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829524.682567:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829524.682584:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829524.682594:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9a7890a3fc00 x1874671476964864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829524.682605:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476964864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829524.682619:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829524.682623:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829524.682627:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476964864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829525.706024:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829526.730009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829527.754003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829528.777980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829529.801903:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829529.801912:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829529.801916:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829529.801926:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3897 00000100:00080000:0.0:1787829529.801935:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829529.801942:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829529.801957:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829529.801973:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476964992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829529.801997:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829529.801999:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829529.802001:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829529.802003:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3897 00000100:00080000:0.0:1787829529.802006:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829529.802008:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829529.802015:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829529.802023:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476965120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829529.802029:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829529.802031:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829529.802032:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829529.802034:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3897 00000100:00080000:0.0:1787829529.802036:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829529.802037:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829529.802039:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829529.802045:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476965248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829529.802051:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829529.802053:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829529.802054:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829529.802057:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3897 00000100:00080000:0.0:1787829529.802059:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829529.802061:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829529.802064:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829529.802069:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476965376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829529.802076:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476964992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829529.802078:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829529.802086:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829529.802088:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829529.802093:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3897 00000100:00080000:0.0:1787829529.802096:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829529.802099:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829529.802111:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829529.802124:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476965504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829529.802133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829529.802169:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9a7890a3de00 x1874671476964992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829529.802181:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476964992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829529.802188:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829529.802190:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829529.802193:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476964992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829529.802197:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829529.802209:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829529.802215:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9a7890a3fc00 x1874671476965120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829529.802221:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476965120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829529.802226:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829529.802227:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829529.802229:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829529.802232:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829529.802240:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829529.802246:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9a7890a3da40 x1874671476965248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829529.802252:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476965248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829529.802256:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829529.802258:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829529.802259:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829529.802275:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829529.802284:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829529.802290:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9a7890a3c3c0 x1874671476965376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829529.802295:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476965376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829529.802300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829529.802302:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829529.802303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829529.802307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829529.802320:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829529.802329:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9a7890a3c000 x1874671476965504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829529.802339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476965504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829529.802348:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829529.802352:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829529.802355:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829530.825978:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829531.849992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829532.873968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829533.897999:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829534.921904:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829534.921913:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829534.921917:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829534.921926:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3902 00000100:00080000:0.0:1787829534.921935:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829534.921942:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829534.921969:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829534.921983:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476965632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829534.922012:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829534.922016:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829534.922017:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829534.922021:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3902 00000100:00080000:0.0:1787829534.922025:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829534.922029:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829534.922039:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476965632:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787829534.922044:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829534.922056:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476965760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829534.922065:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829534.922067:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829534.922068:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829534.922072:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3902 00000100:00080000:0.0:1787829534.922081:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829534.922085:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829534.922089:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829534.922089:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829534.922098:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476965888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829534.922105:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829534.922108:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829534.922109:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829534.922112:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3902 00000100:00080000:0.0:1787829534.922115:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829534.922118:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829534.922122:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829534.922130:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476966016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829534.922132:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9a7890a3f480 x1874671476965632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829534.922138:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829534.922141:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829534.922142:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829534.922146:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3902 00000100:00100000:3.0:1787829534.922147:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476965632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829534.922148:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829534.922151:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829534.922155:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829534.922157:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829534.922160:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787829534.922163:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476966144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829534.922164:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476965632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829534.922184:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829534.922205:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829534.922214:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9a7890a3c000 x1874671476965760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829534.922221:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476965760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829534.922226:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829534.922228:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829534.922230:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829534.922234:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829534.922246:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829534.922256:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9a7890a3c3c0 x1874671476965888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829534.922267:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476965888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829534.922274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829534.922277:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829534.922280:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476965888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829534.922286:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829534.922304:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829534.922315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9a7890a3da40 x1874671476966016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829534.922325:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476966016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829534.922334:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829534.922337:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829534.922340:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829534.922344:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829534.922357:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829534.922364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9a7890a3fc00 x1874671476966144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829534.922371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476966144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829534.922377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829534.922379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829534.922381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829535.945992:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829536.969958:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829537.993966:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829539.017938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829540.041927:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829540.041934:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829540.041937:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829540.041945:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3907 00000100:00080000:2.0:1787829540.041953:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829540.041957:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829540.041975:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829540.041986:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476966272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829540.042004:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829540.042007:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829540.042008:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829540.042010:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3907 00000100:00080000:2.0:1787829540.042013:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829540.042016:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829540.042023:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829540.042033:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476966272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829540.042034:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476966400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829540.042045:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829540.042048:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829540.042049:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829540.042051:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3907 00000100:00080000:2.0:1787829540.042053:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829540.042055:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829540.042058:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829540.042065:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476966528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829540.042071:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829540.042074:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829540.042075:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829540.042077:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3907 00000100:00080000:2.0:1787829540.042084:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829540.042086:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829540.042089:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829540.042090:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829540.042099:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476966656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829540.042106:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829540.042108:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829540.042108:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829540.042111:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3907 00000100:00080000:2.0:1787829540.042113:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829540.042115:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829540.042118:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829540.042127:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476966784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829540.042131:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829540] req@ffff9a78841db840 x1874671476966272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829540.042146:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476966272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829540.042155:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829540.042159:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829540.042163:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476966272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829540.042182:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829540.042203:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829540.042216:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829540] req@ffff9a78841d92c0 x1874671476966400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829540.042228:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476966400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829540.042243:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829540.042245:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829540.042247:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829540.042251:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829540.042265:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829540.042272:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829540] req@ffff9a78841d9680 x1874671476966528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829540.042278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476966528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829540.042283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829540.042285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829540.042287:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829540.042290:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829540.042298:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829540.042303:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829540] req@ffff9a78841d8b40 x1874671476966656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829540.042309:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476966656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829540.042314:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829540.042316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829540.042318:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829540.042321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829540.042329:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829540.042334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829540/real 1787829540] req@ffff9a78841da940 x1874671476966784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829540.042340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476966784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829540.042345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829540.042346:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829540.042348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476966784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829541.065955:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829542.089933:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829543.113970:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829544.137989:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829545.161920:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829545.161929:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829545.161939:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829545.161949:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3912 00000100:00080000:2.0:1787829545.161962:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829545.161971:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829545.161971:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829545.162013:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476966912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829545.162041:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829545.162045:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829545.162047:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829545.162051:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3912 00000100:00080000:2.0:1787829545.162055:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829545.162059:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829545.162067:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829545.162078:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476967040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829545.162085:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476966912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829545.162090:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829545.162094:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829545.162095:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829545.162099:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3912 00000100:00080000:2.0:1787829545.162101:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829545.162103:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829545.162107:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829545.162116:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476967168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829545.162124:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829545.162127:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829545.162128:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829545.162131:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3912 00000100:00080000:2.0:1787829545.162140:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829545.162143:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829545.162146:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829545.162152:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829545.162155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476967296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829545.162163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829545.162166:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829545.162167:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829545.162170:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3912 00000100:00080000:2.0:1787829545.162178:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829545.162180:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829545.162183:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829545.162193:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476967424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829545.162205:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff9a78841da940 x1874671476966912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.162221:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476966912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.162229:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.162232:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829545.162237:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476966912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829545.162255:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829545.162273:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829545.162284:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff9a78841d8b40 x1874671476967040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.162297:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476967040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.162305:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.162309:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829545.162313:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829545.162319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829545.162341:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829545.162350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff9a78841d9680 x1874671476967168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.162358:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476967168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.162364:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.162366:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829545.162368:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829545.162373:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829545.162385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829545.162391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff9a78841d92c0 x1874671476967296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.162398:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476967296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.162405:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.162407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829545.162409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829545.162413:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829545.162421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829545.162427:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829545/real 1787829545] req@ffff9a78841db840 x1874671476967424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829545.162434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476967424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829545.162440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829545.162442:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829545.162444:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829546.186009:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829547.209969:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829548.233990:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829549.257947:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829550.281893:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829550.281902:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829550.281906:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829550.281914:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3917 00000100:00080000:2.0:1787829550.281924:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829550.281930:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829550.281939:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829550.281961:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476967552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829550.281982:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829550.281984:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829550.281986:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829550.281988:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3917 00000100:00080000:2.0:1787829550.281991:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829550.281993:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829550.282001:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829550.282009:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476967680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829550.282017:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829550.282022:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829550.282022:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476967552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829550.282026:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829550.282033:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3917 00000100:00080000:2.0:1787829550.282036:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829550.282038:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829550.282042:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829550.282050:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476967808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829550.282058:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829550.282061:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829550.282062:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829550.282065:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3917 00000100:00080000:2.0:1787829550.282069:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829550.282071:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829550.282074:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829550.282079:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829550.282083:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476967936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829550.282092:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829550.282096:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829550.282097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829550.282101:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3917 00000100:00080000:2.0:1787829550.282105:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829550.282107:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829550.282111:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829550.282122:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476968064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829550.282136:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9a78841db480 x1874671476967552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829550.282151:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476967552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829550.282159:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829550.282162:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829550.282165:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476967552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829550.282182:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829550.282200:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829550.282213:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9a78841d83c0 x1874671476967680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829550.282226:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476967680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829550.282236:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829550.282240:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829550.282243:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829550.282250:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829550.282271:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829550.282283:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9a78841d8000 x1874671476967808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829550.282296:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476967808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829550.282304:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829550.282307:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829550.282311:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829550.282318:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829550.282338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829550.282349:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9a78841da580 x1874671476967936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829550.282361:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476967936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829550.282371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829550.282374:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829550.282378:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476967936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829550.282384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829550.282401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829550.282409:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9a78841d9e00 x1874671476968064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829550.282417:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476968064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829550.282424:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829550.282426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829550.282428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829551.305931:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829552.329935:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829553.353951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829554.377965:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829555.401929:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829555.401941:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829555.401945:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829555.401956:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3922 00000100:00080000:2.0:1787829555.401966:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829555.401974:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829555.402011:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476968192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829555.402033:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829555.402036:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829555.402038:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829555.402041:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3922 00000100:00080000:2.0:1787829555.402050:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829555.402053:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829555.402061:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829555.402069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476968320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829555.402076:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829555.402078:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829555.402079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829555.402082:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3922 00000100:00080000:2.0:1787829555.402083:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829555.402085:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829555.402088:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829555.402093:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476968448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829555.402100:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829555.402103:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829555.402105:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829555.402108:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3922 00000100:00080000:2.0:1787829555.402111:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829555.402115:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829555.402120:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829555.402130:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476968576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829555.402140:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829555.402143:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829555.402145:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829555.402149:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3922 00000100:00080000:2.0:1787829555.402160:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829555.402164:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829555.402169:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829555.402179:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476968704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829555.402330:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476968192:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829555.402369:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:3.0:1787829555.402381:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829555.402440:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9a78841d8780 x1874671476968192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.402459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476968192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.402469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.402474:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829555.402478:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476968192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829555.402485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829555.402510:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829555.402524:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9a78841d9a40 x1874671476968320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.402537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476968320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.402545:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.402548:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829555.402552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829555.402557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829555.402580:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829555.402592:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9a78841da1c0 x1874671476968448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.402604:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476968448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.402613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.402616:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829555.402620:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829555.402626:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829555.402646:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829555.402657:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9a78841db0c0 x1874671476968576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.402676:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476968576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.402687:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.402690:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829555.402694:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829555.402700:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829555.402718:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829555.402727:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9a78841dbc00 x1874671476968704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829555.402737:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476968704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829555.402747:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829555.402751:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829555.402754:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829556.425963:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829557.450012:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829558.473971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829559.498018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829560.521940:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829560.521951:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829560.521955:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829560.521966:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3928 00000100:00080000:1.0:1787829560.521978:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829560.521987:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829560.522025:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476968832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829560.522047:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829560.522051:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829560.522054:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829560.522062:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3928 00000100:00100000:3.0:1787829560.522063:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476968832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829560.522067:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829560.522074:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829560.522084:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829560.522100:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476968960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829560.522112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829560.522115:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829560.522117:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829560.522124:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829560.522128:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3928 00000100:00080000:1.0:1787829560.522135:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829560.522140:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829560.522146:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829560.522161:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476969088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829560.522172:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829560.522176:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829560.522179:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829560.522183:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3928 00000100:00080000:1.0:1787829560.522189:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829560.522189:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9a7890b1e940 x1874671476968832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829560.522199:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829560.522208:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829560.522219:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476968832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829560.522224:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476969216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829560.522241:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829560.522245:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829560.522246:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829560.522252:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829560.522254:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829560.522262:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476968832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829560.522265:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3928 00000100:00080000:1.0:1787829560.522271:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829560.522276:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829560.522281:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829560.522286:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829560.522296:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476969344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829560.522309:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829560.522323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9a7890b1e580 x1874671476968960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829560.522332:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476968960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829560.522339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829560.522341:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829560.522350:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476968960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829560.522357:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829560.522369:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829560.522376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9a7890b1de00 x1874671476969088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829560.522401:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476969088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829560.522411:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829560.522413:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829560.522416:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829560.522420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829560.522435:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829560.522442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9a7890b1d680 x1874671476969216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829560.522449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476969216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829560.522456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829560.522458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829560.522460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829560.522471:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829560.522482:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829560.522488:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9a7890b1da40 x1874671476969344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829560.522495:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476969344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829560.522501:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829560.522503:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829560.522505:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829560.522553:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829561.545989:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829562.569953:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829563.593990:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829564.618013:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829565.642021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829565.642049:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829565.642059:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829565.642064:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829565.642074:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3933 00000100:00080000:1.0:1787829565.642087:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829565.642096:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829565.642153:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476969472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829565.642185:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829565.642188:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829565.642189:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829565.642192:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3933 00000100:00080000:1.0:1787829565.642195:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829565.642199:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829565.642209:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829565.642223:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476969600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829565.642232:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829565.642237:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829565.642241:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829565.642243:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476969472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829565.642247:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3933 00000100:00080000:1.0:1787829565.642254:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829565.642262:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829565.642265:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829565.642275:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476969728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829565.642282:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829565.642285:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829565.642286:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829565.642289:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3933 00000100:00080000:1.0:1787829565.642292:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829565.642296:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829565.642300:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829565.642309:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476969856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829565.642318:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829565.642321:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829565.642322:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829565.642325:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3933 00000100:00080000:1.0:1787829565.642329:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829565.642335:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829565.642336:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829565.642359:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829565.642373:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476969984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829565.642417:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9a7890b1c000 x1874671476969472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829565.642439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476969472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829565.642450:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829565.642455:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829565.642459:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476969472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829565.642493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829565.642519:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829565.642532:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9a7890b1f840 x1874671476969600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829565.642544:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476969600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829565.642553:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829565.642557:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829565.642560:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829565.642567:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829565.642592:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829565.642604:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9a7890b1fc00 x1874671476969728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829565.642616:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476969728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829565.642624:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829565.642629:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829565.642632:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829565.642639:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829565.642659:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829565.642670:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9a7890b1f0c0 x1874671476969856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829565.642682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476969856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829565.642701:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829565.642705:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829565.642708:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829565.642715:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829565.642734:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829565.642753:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9a7890b1f480 x1874671476969984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829565.642766:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476969984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829565.642777:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829565.642780:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829565.642784:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476969984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829566.665955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829567.690002:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829568.713981:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829569.737999:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829570.761934:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829570.761944:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829570.761949:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829570.761967:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3938 00000100:00080000:0.0:1787829570.761978:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787829570.761985:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829570.761986:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829570.762024:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476970112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829570.762049:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829570.762055:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829570.762056:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829570.762059:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3938 00000100:00080000:0.0:1787829570.762061:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829570.762065:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829570.762074:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829570.762086:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476970240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829570.762087:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476970112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829570.762101:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829570.762104:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829570.762105:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829570.762108:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3938 00000100:00080000:0.0:1787829570.762111:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829570.762113:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829570.762117:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829570.762126:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476970368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829570.762132:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829570.762135:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829570.762136:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829570.762138:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3938 00000100:00080000:0.0:1787829570.762141:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829570.762143:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829570.762146:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829570.762150:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829570.762154:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476970496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829570.762162:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829570.762165:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829570.762166:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829570.762168:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3938 00000100:00080000:0.0:1787829570.762170:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829570.762173:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829570.762176:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829570.762183:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476970624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829570.762194:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9a7890a3de00 x1874671476970112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829570.762209:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476970112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829570.762218:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829570.762221:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829570.762224:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476970112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829570.762253:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829570.762268:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829570.762276:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9a7890a3fc00 x1874671476970240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829570.762284:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476970240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829570.762290:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829570.762292:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829570.762294:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829570.762298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829570.762321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829570.762331:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9a7890a3da40 x1874671476970368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829570.762340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476970368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829570.762345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829570.762348:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829570.762350:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829570.762354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829570.762371:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829570.762381:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9a7890a3c3c0 x1874671476970496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829570.762394:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476970496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829570.762404:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829570.762416:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829570.762420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829570.762425:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829570.762439:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829570.762447:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9a7890a3c000 x1874671476970624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829570.762455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476970624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829570.762462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829570.762464:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829570.762466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829571.785951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829572.810024:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829573.833992:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829574.858018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829575.881948:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829575.881956:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829575.881960:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787829575.881962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829575.881970:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3943 00000100:00080000:2.0:1787829575.881982:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829575.881989:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829575.882021:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476970752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829575.882043:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829575.882045:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829575.882047:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829575.882050:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3943 00000100:00080000:2.0:1787829575.882053:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829575.882055:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829575.882063:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829575.882071:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476970880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829575.882079:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476970752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829575.882082:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829575.882087:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829575.882089:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829575.882092:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3943 00000100:00080000:2.0:1787829575.882094:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829575.882097:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829575.882101:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829575.882108:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476971008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829575.882114:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829575.882117:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829575.882119:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829575.882122:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3943 00000100:00080000:2.0:1787829575.882131:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829575.882134:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829575.882138:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829575.882139:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829575.882150:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476971136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829575.882158:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829575.882160:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829575.882161:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829575.882164:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3943 00000100:00080000:2.0:1787829575.882166:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829575.882169:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829575.882172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829575.882181:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476971264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829575.882184:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829575] req@ffff9a78841dbc00 x1874671476970752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829575.882212:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476970752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829575.882220:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829575.882224:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829575.882226:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476970752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829575.882245:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829575.882260:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829575.882268:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829575] req@ffff9a78841db0c0 x1874671476970880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829575.882291:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476970880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829575.882302:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829575.882304:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829575.882306:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476970880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829575.882311:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829575.882325:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829575.882332:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829575] req@ffff9a78841da1c0 x1874671476971008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829575.882340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476971008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829575.882345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829575.882347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829575.882348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829575.882352:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829575.882372:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829575.882379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829575] req@ffff9a78841d9a40 x1874671476971136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829575.882386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476971136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829575.882392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829575.882394:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829575.882397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829575.882401:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829575.882410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829575.882422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829575] req@ffff9a78841d8780 x1874671476971264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829575.882429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476971264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829575.882436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829575.882437:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829575.882439:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829576.906026:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829577.929992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829578.954061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829579.977946:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829581.001960:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.001969:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829581.001972:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.001981:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3948 00000100:00080000:2.0:1787829581.001991:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.001998:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829581.002029:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476971392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829581.002037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829581.002062:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.002066:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829581.002068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.002071:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3948 00000100:00080000:2.0:1787829581.002074:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.002077:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829581.002085:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829581.002098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476971520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829581.002110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.002120:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829581.002120:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476971392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829581.002125:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.002138:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3948 00000100:00080000:2.0:1787829581.002143:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.002148:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829581.002154:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829581.002167:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476971648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829581.002175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.002178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829581.002179:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.002183:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3948 00000100:00080000:2.0:1787829581.002200:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.002203:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829581.002206:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829581.002211:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829581.002216:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476971776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829581.002224:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829581.002227:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829581.002231:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829581.002234:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3948 00000100:00080000:2.0:1787829581.002254:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829581.002259:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829581.002265:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829581.002279:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476971904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829581.002300:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9a78841d9e00 x1874671476971392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.002319:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476971392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.002329:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.002332:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829581.002336:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476971392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829581.002363:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829581.002383:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829581.002394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9a78841da580 x1874671476971520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.002402:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476971520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.002408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.002411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829581.002413:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829581.002417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829581.002428:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829581.002434:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9a78841d8000 x1874671476971648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.002442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476971648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.002447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.002449:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829581.002451:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829581.002454:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829581.002465:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829581.002471:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9a78841d83c0 x1874671476971776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.002477:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476971776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.002485:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.002487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829581.002489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829581.002493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829581.002502:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829581.002507:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829581/real 1787829581] req@ffff9a78841db480 x1874671476971904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829581.002514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476971904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829581.002521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829581.002523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829581.002525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476971904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829582.025960:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829583.049995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829584.074049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829585.097975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829586.121920:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829586.121931:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829586.121936:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829586.121947:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3953 00010000:00010000:2.0:1787829586.121953:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829586.121959:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829586.121967:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829586.122005:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476972032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829586.122034:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829586.122041:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829586.122042:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829586.122046:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3953 00000100:00080000:1.0:1787829586.122050:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829586.122055:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829586.122065:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829586.122074:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476972032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829586.122081:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476972160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829586.122094:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829586.122098:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829586.122100:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829586.122104:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3953 00000100:00080000:1.0:1787829586.122108:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829586.122112:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829586.122117:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829586.122129:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476972288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829586.122139:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829586.122143:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829586.122145:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829586.122149:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3953 00000100:00080000:1.0:1787829586.122154:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829586.122159:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829586.122165:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829586.122176:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829586.122182:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476972416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829586.122194:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829586.122198:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829586.122200:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829586.122217:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972288:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829586.122223:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3953 00000100:00080000:1.0:1787829586.122229:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829586.122234:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829586.122239:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829586.122247:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787829586.122256:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476972544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829586.122273:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829587.145991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829588.169959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829589.194084:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829589.194176:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787829589.194232:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829589.194247:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829589.194301:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829589.194310:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829589.194318:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829589.194326:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829589.194393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829589] req@ffff9a7890b1c000 x1874671476972544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829589.194426:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476972544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829589.194441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829589.194446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829589.194451:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972544:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829589.194459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829589] req@ffff9a7890b1f840 x1874671476972416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829589.194471:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476972416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829589.194482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829589.194485:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829589.194489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829589.194504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829589] req@ffff9a7890b1f0c0 x1874671476972160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829589.194516:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476972160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829589.194525:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829589.194528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829589.194532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829589.194539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829589] req@ffff9a7890b1fc00 x1874671476972288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829589.194556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476972288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829589.194565:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829589.194568:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829589.194571:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829589.194578:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829586/real 1787829589] req@ffff9a7890b1f480 x1874671476972032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829589.194591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476972032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829589.194598:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829589.194602:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829589.194606:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476972032:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829590.218018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829591.241929:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829591.241938:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829591.241941:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829591.241949:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3958 00000100:00080000:1.0:1787829591.241963:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829591.241969:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829591.242001:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476972672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829591.242012:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829591.242024:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829591.242028:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829591.242030:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829591.242033:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3958 00000100:00080000:1.0:1787829591.242035:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829591.242038:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829591.242045:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829591.242054:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476972800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829591.242061:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829591.242063:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829591.242065:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829591.242067:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3958 00000100:00080000:1.0:1787829591.242069:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829591.242072:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829591.242075:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829591.242084:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476972928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829591.242090:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829591.242092:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829591.242094:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829591.242096:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3958 00000100:00080000:1.0:1787829591.242099:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829591.242102:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829591.242106:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829591.242112:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476973056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829591.242119:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829591.242120:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476972672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829591.242123:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829591.242127:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829591.242131:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3958 00000100:00080000:1.0:1787829591.242133:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829591.242135:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829591.242138:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829591.242146:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476973184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829591.242196:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829591.242250:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9a7890b1da40 x1874671476972672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829591.242269:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476972672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829591.242280:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829591.242284:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829591.242289:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476972672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829591.242297:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829591.242326:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829591.242346:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9a7890b1d680 x1874671476972800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829591.242358:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476972800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829591.242363:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829591.242366:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829591.242368:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829591.242372:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829591.242388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829591.242399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9a7890b1de00 x1874671476972928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829591.242411:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476972928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829591.242419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829591.242423:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829591.242427:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476972928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829591.242455:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829591.242480:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829591.242493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9a7890b1e580 x1874671476973056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829591.242505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476973056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829591.242516:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829591.242519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829591.242522:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829591.242528:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829591.242546:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829591.242557:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829591/real 1787829591] req@ffff9a7890b1e940 x1874671476973184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829591.242568:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476973184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829591.242579:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829591.242582:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829591.242585:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829592.266071:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829593.290012:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829594.313983:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829595.337999:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829596.361939:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.361949:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829596.361953:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.361963:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3963 00000100:00080000:3.0:1787829596.361972:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.361978:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829596.362009:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671476973312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829596.362037:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.362041:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829596.362042:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.362046:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3963 00000100:00080000:3.0:1787829596.362049:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.362053:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829596.362063:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829596.362076:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476973312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829596.362077:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671476973440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829596.362091:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.362095:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829596.362096:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.362100:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3963 00000100:00080000:3.0:1787829596.362105:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.362108:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829596.362113:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829596.362126:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829596.362128:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476973568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829596.362150:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.362154:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829596.362157:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.362161:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3963 00000100:00080000:3.0:1787829596.362165:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.362169:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829596.362174:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829596.362186:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476973696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829596.362195:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff9a78a67743c0 x1874671476973312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829596.362196:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829596.362205:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829596.362209:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829596.362219:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3963 00000100:00080000:3.0:1787829596.362223:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829596.362227:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829596.362228:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671476973312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1787829596.362235:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787829596.362239:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829596.362242:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829596.362246:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476973312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829596.362250:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476973824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829596.362268:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829596.362285:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829596.362295:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff9a78a6777c00 x1874671476973440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829596.362304:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671476973440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829596.362309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829596.362312:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829596.362314:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829596.362319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973568:192.168.203.140@tcp:38:kworker.0 00010000:00010000:0.0:1787829596.362330:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:2.0:1787829596.362335:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829596.362344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff9a78a6774f00 x1874671476973568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829596.362350:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476973568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829596.362359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829596.362361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829596.362362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829596.362365:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829596.362374:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829596.362379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff9a78a6777840 x1874671476973696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829596.362389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476973696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829596.362398:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829596.362401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829596.362404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829596.362410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829596.362427:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829596.362437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829596/real 1787829596] req@ffff9a78a6775e00 x1874671476973824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829596.362447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476973824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829596.362456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829596.362459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829596.362462:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476973824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829597.385968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829598.409995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829599.433946:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829600.457978:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829601.481918:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829601.481924:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829601.481928:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829601.481936:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3968 00000100:00080000:1.0:1787829601.481944:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829601.481949:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829601.481958:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829601.481978:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476973952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829601.481999:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829601.482002:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829601.482003:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829601.482005:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3968 00000100:00080000:1.0:1787829601.482007:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829601.482010:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829601.482017:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829601.482025:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476973952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829601.482027:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476974080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829601.482036:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829601.482038:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829601.482039:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829601.482042:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3968 00000100:00080000:1.0:1787829601.482044:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829601.482046:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829601.482048:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829601.482055:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476974208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829601.482061:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829601.482063:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829601.482064:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829601.482066:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3968 00000100:00080000:1.0:1787829601.482069:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829601.482071:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829601.482074:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829601.482075:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829601.482083:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476974336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829601.482090:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829601.482092:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829601.482093:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829601.482107:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3968 00000100:00080000:1.0:1787829601.482111:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829601.482114:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829601.482118:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829601.482127:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829601/real 1787829601] req@ffff9a7890b1e940 x1874671476973952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829601.482129:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476974464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829601.482151:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476973952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829601.482163:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829601.482167:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829601.482169:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476973952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829601.482186:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829601.482201:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829601.482209:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829601/real 1787829601] req@ffff9a7890b1e580 x1874671476974080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829601.482216:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476974080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829601.482221:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829601.482222:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829601.482225:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829601.482230:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829601.482246:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829601.482265:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829601/real 1787829601] req@ffff9a7890b1de00 x1874671476974208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829601.482278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476974208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829601.482283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829601.482285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829601.482287:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829601.482290:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829601.482304:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829601.482311:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829601/real 1787829601] req@ffff9a7890b1d680 x1874671476974336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829601.482321:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476974336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829601.482329:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829601.482333:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829601.482336:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829601.482341:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829601.482359:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829601.482369:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829601/real 1787829601] req@ffff9a7890b1da40 x1874671476974464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829601.482386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476974464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829601.482395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829601.482398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829601.482401:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829602.505984:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829603.529946:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829604.554006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829605.577986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829606.601933:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829606.601942:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829606.601945:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829606.601953:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3974 00000100:00080000:3.0:1787829606.601962:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829606.601967:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829606.601998:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476974592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829606.602019:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829606.602023:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829606.602024:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829606.602026:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3974 00000100:00080000:3.0:1787829606.602029:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829606.602031:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829606.602038:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829606.602047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476974720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829606.602055:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829606.602057:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829606.602069:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829606.602073:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3974 00000100:00080000:3.0:1787829606.602076:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829606.602078:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829606.602084:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476974592:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787829606.602085:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829606.602099:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476974848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829606.602107:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829606.602109:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829606.602111:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829606.602113:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3974 00000100:00080000:3.0:1787829606.602115:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829606.602118:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829606.602123:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829606.602130:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476974976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829606.602136:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829606.602138:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829606.602140:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829606.602142:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3974 00000100:00080000:3.0:1787829606.602144:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829606.602146:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829606.602149:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829606.602154:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829606.602160:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476975104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829606.602221:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829606/real 1787829606] req@ffff9a78a6776d00 x1874671476974592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787829606.602237:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829606.602244:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476974592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829606.602257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829606.602262:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829606.602267:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476974592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829606.602276:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829606.602301:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829606.602315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829606/real 1787829606] req@ffff9a78a67761c0 x1874671476974720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829606.602329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476974720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829606.602345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829606.602349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829606.602352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829606.602378:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829606.602400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829606.602412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829606/real 1787829606] req@ffff9a78a6774780 x1874671476974848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829606.602425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476974848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829606.602433:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829606.602436:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829606.602440:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829606.602447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829606.602467:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829606.602479:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829606/real 1787829606] req@ffff9a78a6775680 x1874671476974976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829606.602491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476974976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829606.602501:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829606.602505:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829606.602509:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476974976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829606.602515:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829606.602546:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829606.602558:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829606/real 1787829606] req@ffff9a78a67770c0 x1874671476975104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829606.602571:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476975104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829606.602580:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829606.602584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829606.602588:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829607.626000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829608.650006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829609.674082:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829610.698086:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829611.721987:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829611.722002:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829611.722008:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829611.722040:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3979 00000100:00080000:1.0:1787829611.722054:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829611.722061:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829611.722066:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829611.722106:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476975232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829611.722131:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829611.722136:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829611.722137:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829611.722141:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3979 00000100:00080000:1.0:1787829611.722145:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829611.722150:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829611.722162:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829611.722178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476975360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829611.722190:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829611.722194:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829611.722196:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829611.722202:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3979 00000100:00080000:1.0:1787829611.722207:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829611.722211:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829611.722221:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829611.722221:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476975232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829611.722253:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476975488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829611.722266:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829611.722270:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829611.722273:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829611.722277:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3979 00000100:00080000:1.0:1787829611.722285:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787829611.722293:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829611.722302:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829611.722310:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829611.722331:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476975616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829611.722343:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829611.722347:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829611.722349:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829611.722355:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829611/real 1787829611] req@ffff9a7890b1f480 x1874671476975232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829611.722360:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3979 00000100:00080000:1.0:1787829611.722371:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829611.722384:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829611.722390:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476975232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829611.722391:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:2.0:1787829611.722409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829611.722415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787829611.722416:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476975744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829611.722422:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476975232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829611.722437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829611.722464:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829611.722479:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829611/real 1787829611] req@ffff9a7890b1fc00 x1874671476975360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829611.722488:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476975360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829611.722493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829611.722496:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829611.722498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829611.722518:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829611.722530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829611.722536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829611/real 1787829611] req@ffff9a7890b1f0c0 x1874671476975488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829611.722543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476975488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829611.722548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829611.722550:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829611.722552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829611.722556:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829611.722566:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829611.722571:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829611/real 1787829611] req@ffff9a7890b1f840 x1874671476975616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829611.722578:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476975616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829611.722585:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829611.722587:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829611.722589:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829611.722593:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829611.722602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829611.722607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829611/real 1787829611] req@ffff9a7890b1c000 x1874671476975744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829611.722614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476975744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829611.722620:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829611.722622:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829611.722624:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476975744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829612.746033:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829613.770075:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829614.794045:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829615.817956:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829616.841910:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829616.841925:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829616.841931:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829616.841944:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3984 00000100:00080000:3.0:1787829616.841958:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829616.841968:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829616.842035:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476975872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829616.842071:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829616.842076:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829616.842077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829616.842082:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3984 00000100:00080000:3.0:1787829616.842087:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829616.842092:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829616.842105:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829616.842122:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476976000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829616.842133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829616.842137:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829616.842139:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829616.842146:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3984 00000100:00080000:3.0:1787829616.842150:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829616.842155:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829616.842162:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829616.842179:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476975872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829616.842182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476976128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829616.842206:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829616.842211:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829616.842213:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829616.842220:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3984 00000100:00080000:3.0:1787829616.842225:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829616.842232:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829616.842238:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829616.842257:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476976256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829616.842262:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787829616.842285:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829616.842289:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829616.842291:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829616.842297:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3984 00000100:00080000:3.0:1787829616.842303:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829616.842308:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829616.842315:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829616.842337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829616/real 1787829616] req@ffff9a78a6776940 x1874671476975872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829616.842338:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476976384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829616.842405:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476975872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829616.842422:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829616.842427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829616.842431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476975872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829616.842439:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829616.842467:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:0.0:1787829616.842470:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829616.842487:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829616/real 1787829616] req@ffff9a78a6776580 x1874671476976000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829616.842514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476976000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829616.842522:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829616.842525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829616.842528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829616.842550:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829616.842567:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829616.842575:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829616/real 1787829616] req@ffff9a78a6774b40 x1874671476976128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829616.842583:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476976128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829616.842589:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829616.842591:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829616.842593:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829616.842598:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829616.842609:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829616.842615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829616/real 1787829616] req@ffff9a78a67752c0 x1874671476976256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829616.842623:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476976256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829616.842629:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829616.842631:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829616.842633:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829616.842636:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829616.842646:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829616.842651:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829616/real 1787829616] req@ffff9a78a6775a40 x1874671476976384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829616.842659:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476976384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829616.842665:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829616.842666:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829616.842668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829617.865961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829618.889941:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829619.913945:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829620.937973:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829621.961933:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829621.961941:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829621.961944:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829621.961968:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3989 00000100:00080000:1.0:1787829621.961978:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829621.961986:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829621.962000:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829621.962022:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476976512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829621.962044:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829621.962047:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829621.962049:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829621.962052:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3989 00000100:00080000:1.0:1787829621.962055:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829621.962058:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829621.962066:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829621.962076:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476976640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829621.962084:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829621.962086:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829621.962088:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829621.962091:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3989 00000100:00080000:1.0:1787829621.962094:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829621.962096:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829621.962100:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829621.962105:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476976512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829621.962111:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476976768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829621.962121:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829621.962124:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829621.962125:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829621.962128:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3989 00000100:00080000:1.0:1787829621.962131:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829621.962133:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829621.962137:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829621.962146:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476976896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829621.962157:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829621.962160:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829621.962161:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829621.962165:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3989 00000100:00080000:1.0:1787829621.962167:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829621.962170:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829621.962173:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829621.962179:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829621.962183:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476977024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829621.962235:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829621/real 1787829621] req@ffff9a7890b1c000 x1874671476976512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829621.962254:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476976512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829621.962265:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829621.962270:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829621.962275:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476976512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829621.962284:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829621.962308:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829621.962323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829621/real 1787829621] req@ffff9a7890b1f840 x1874671476976640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829621.962336:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476976640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829621.962345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829621.962350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829621.962353:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829621.962379:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829621.962401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829621.962414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829621/real 1787829621] req@ffff9a7890b1f0c0 x1874671476976768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829621.962426:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476976768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829621.962447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829621.962451:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829621.962456:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829621.962463:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829621.962484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829621.962497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829621/real 1787829621] req@ffff9a7890b1fc00 x1874671476976896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829621.962510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476976896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829621.962521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829621.962525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829621.962529:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476976896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829621.962535:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829621.962554:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829621.962564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829621/real 1787829621] req@ffff9a7890b1f480 x1874671476977024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829621.962577:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476977024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829646 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829621.962599:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829621.962602:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829621.962606:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829622.985958:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829624.010043:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829625.033993:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829626.057967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829627.081954:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829627.081962:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829627.081966:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829627.081975:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3994 00000100:00080000:1.0:1787829627.081983:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829627.081988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829627.081989:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829627.082024:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476977152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829627.082045:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829627.082048:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829627.082052:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829627.082056:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3994 00000100:00080000:1.0:1787829627.082058:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829627.082061:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829627.082069:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829627.082078:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476977280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829627.082086:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829627.082089:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829627.082092:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829627.082093:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476977152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829627.082096:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3994 00000100:00080000:1.0:1787829627.082101:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829627.082105:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829627.082108:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829627.082116:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476977408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829627.082123:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829627.082126:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829627.082127:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829627.082129:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3994 00000100:00080000:1.0:1787829627.082155:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829627.082158:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829627.082161:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829627.082165:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829627.082172:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476977536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829627.082180:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829627.082183:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829627.082184:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829627.082188:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3994 00000100:00080000:1.0:1787829627.082190:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829627.082193:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829627.082196:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829627.082205:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476977664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829627.082227:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9a7890b1da40 x1874671476977152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.082248:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476977152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.082257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.082260:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829627.082262:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476977152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829627.082282:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829627.082300:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829627.082309:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9a7890b1d680 x1874671476977280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.082318:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476977280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.082324:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.082326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829627.082328:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829627.082332:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829627.082342:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829627.082350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9a7890b1de00 x1874671476977408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.082370:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476977408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.082385:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.082389:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829627.082393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829627.082400:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829627.082418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829627.082426:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9a7890b1e580 x1874671476977536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.082434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476977536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.082441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.082443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829627.082445:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829627.082449:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829627.082458:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829627.082464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9a7890b1e940 x1874671476977664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829627.082482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476977664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829627.082488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829627.082490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829627.082492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829628.105955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829629.130010:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829630.153980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829631.178037:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829632.202018:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829632.202071:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829632.202078:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829632.202082:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829632.202092:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 3999 00000100:00080000:1.0:1787829632.202101:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829632.202107:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829632.202144:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476977792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829632.202169:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829632.202173:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829632.202174:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829632.202178:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3999 00000100:00080000:1.0:1787829632.202181:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829632.202185:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829632.202194:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829632.202207:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476977920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829632.202223:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829632.202227:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829632.202227:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476977792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829632.202231:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829632.202239:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3999 00000100:00080000:1.0:1787829632.202243:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829632.202246:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829632.202251:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829632.202262:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476978048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829632.202272:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829632.202284:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829632.202286:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829632.202290:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3999 00000100:00080000:1.0:1787829632.202294:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787829632.202300:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829632.202302:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829632.202309:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829632.202321:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476978176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829632.202331:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829632.202335:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829632.202337:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829632.202341:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 3999 00000100:00080000:1.0:1787829632.202345:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829632.202349:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829632.202354:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829632.202365:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476978304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829632.202372:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff9a7890b1e940 x1874671476977792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.202396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476977792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.202409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.202414:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829632.202418:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476977792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829632.202444:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829632.202471:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829632.202485:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff9a7890b1e580 x1874671476977920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.202499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476977920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.202509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.202512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829632.202516:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476977920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829632.202523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829632.202545:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829632.202559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff9a7890b1de00 x1874671476978048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.202572:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476978048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.202581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.202585:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829632.202589:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829632.202595:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829632.202616:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829632.202629:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff9a7890b1d680 x1874671476978176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.202642:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476978176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.202653:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.202656:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829632.202660:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829632.202667:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829632.202689:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829632.202702:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829632/real 1787829632] req@ffff9a7890b1da40 x1874671476978304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829632.202715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476978304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829632.202725:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829632.202729:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829632.202732:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829633.226032:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829634.250040:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829635.274015:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829636.298084:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829637.321971:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829637.321985:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829637.321990:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829637.322003:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4004 00000100:00080000:0.0:1787829637.322017:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829637.322027:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829637.322045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829637.322077:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476978432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829637.322103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829637.322106:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829637.322108:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829637.322112:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4004 00000100:00080000:0.0:1787829637.322117:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829637.322122:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829637.322132:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829637.322144:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476978432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829637.322146:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476978560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829637.322171:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829637.322177:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829637.322179:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829637.322187:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4004 00000100:00080000:0.0:1787829637.322192:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829637.322198:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829637.322204:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829637.322224:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476978688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829637.322225:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829637.322238:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829637.322241:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829637.322244:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829637.322247:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4004 00000100:00080000:0.0:1787829637.322252:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829637.322255:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829637.322259:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829637.322268:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476978816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829637.322275:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829637.322279:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829637.322280:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829637.322284:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4004 00000100:00080000:0.0:1787829637.322286:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829637.322293:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829637.322296:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829637.322301:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff9a7890a3f480 x1874671476978432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829637.322304:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476978944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829637.322353:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476978432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.322369:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.322373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829637.322376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476978432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829637.322397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829637.322419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829637.322429:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff9a7890a3c000 x1874671476978560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829637.322437:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476978560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.322442:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.322445:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829637.322448:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829637.322453:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829637.322464:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829637.322470:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff9a7890a3c3c0 x1874671476978688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829637.322477:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476978688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.322482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.322484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829637.322486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829637.322490:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829637.322501:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829637.322506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff9a7890a3da40 x1874671476978816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829637.322514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476978816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.322520:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.322522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829637.322524:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829637.322528:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829637.322537:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829637.322543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829637/real 1787829637] req@ffff9a7890a3fc00 x1874671476978944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829637.322550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476978944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829637.322556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829637.322558:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829637.322560:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476978944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829638.345985:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829639.369996:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829640.393962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829641.417983:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829642.441931:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829642.441939:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829642.441943:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829642.441953:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4009 00000100:00080000:0.0:1787829642.441962:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787829642.441964:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829642.441970:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829642.442002:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476979072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829642.442022:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829642.442026:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829642.442027:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829642.442029:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4009 00000100:00080000:0.0:1787829642.442032:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829642.442035:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829642.442042:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829642.442052:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476979072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829642.442054:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476979200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829642.442066:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829642.442068:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829642.442069:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829642.442072:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4009 00000100:00080000:0.0:1787829642.442074:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829642.442076:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829642.442079:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829642.442088:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476979328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829642.442095:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829642.442097:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829642.442098:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829642.442100:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4009 00000100:00080000:0.0:1787829642.442105:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829642.442107:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829642.442112:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:2.0:1787829642.442115:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829642.442123:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476979456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829642.442131:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829642.442134:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829642.442135:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829642.442137:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4009 00000100:00080000:0.0:1787829642.442139:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829642.442142:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829642.442145:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829642.442154:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476979584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829642.442159:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff9a7890a3de00 x1874671476979072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.442176:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476979072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.442185:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.442189:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829642.442193:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476979072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829642.442211:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829642.442232:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829642.442246:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff9a7890a3fc00 x1874671476979200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.442260:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476979200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.442269:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.442272:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829642.442275:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829642.442279:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829642.442295:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829642.442302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff9a7890a3da40 x1874671476979328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.442310:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476979328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.442316:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.442318:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829642.442320:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829642.442323:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829642.442333:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829642.442338:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff9a7890a3c3c0 x1874671476979456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.442345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476979456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.442358:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.442361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829642.442365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829642.442371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829642.442392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829642.442403:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829642/real 1787829642] req@ffff9a7890a3c000 x1874671476979584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829642.442414:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476979584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829642.442421:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829642.442423:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829642.442425:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829643.466016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829644.489979:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829645.514082:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829646.538078:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829647.561937:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829647.561945:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829647.561949:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829647.561957:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4015 00000100:00080000:0.0:1787829647.561966:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829647.561972:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829647.562003:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476979712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829647.562026:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829647.562029:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829647.562030:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829647.562033:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4015 00000100:00080000:0.0:1787829647.562044:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787829647.562044:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829647.562054:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829647.562060:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476979712:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787829647.562062:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829647.562077:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476979840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829647.562085:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829647.562088:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829647.562089:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829647.562092:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4015 00000100:00080000:0.0:1787829647.562111:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829647.562113:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829647.562116:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829647.562130:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476979968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829647.562131:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829647.562140:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829647.562143:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829647.562145:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829647.562148:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4015 00000100:00080000:0.0:1787829647.562151:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829647.562154:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829647.562157:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829647.562164:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476980096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829647.562179:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829647.562182:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829647.562183:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829647.562188:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4015 00000100:00100000:3.0:1787829647.562190:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9a7890a3f480 x1874671476979712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829647.562191:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829647.562197:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829647.562203:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829647.562215:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476980224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829647.562222:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476979712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.562238:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.562242:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829647.562246:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476979712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829647.562268:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829647.562302:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829647.562315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9a7890a3c000 x1874671476979840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829647.562329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476979840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.562337:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.562341:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829647.562344:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829647.562351:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829647.562368:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829647.562377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9a7890a3c3c0 x1874671476979968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829647.562389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476979968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.562396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.562399:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829647.562403:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476979968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829647.562408:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829647.562426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829647.562438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9a7890a3da40 x1874671476980096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829647.562449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476980096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.562465:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.562468:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829647.562472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829647.562478:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829647.562506:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829647.562517:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9a7890a3fc00 x1874671476980224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829647.562529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476980224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.562538:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.562541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829647.562545:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829648.586020:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829649.610105:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829650.634016:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829651.657989:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829652.681950:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829652.681961:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829652.681966:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829652.681976:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4020 00000100:00080000:0.0:1787829652.681988:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829652.681995:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829652.682031:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476980352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829652.682058:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829652.682062:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829652.682065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829652.682072:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4020 00000100:00080000:0.0:1787829652.682076:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829652.682082:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829652.682094:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787829652.682098:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787829652.682109:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476980352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829652.682113:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476980480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829652.682128:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829652.682133:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829652.682135:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829652.682139:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4020 00000100:00080000:0.0:1787829652.682144:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829652.682149:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829652.682155:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829652.682168:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476980608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829652.682178:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829652.682182:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829652.682184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829652.682188:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4020 00000100:00080000:0.0:1787829652.682192:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829652.682196:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829652.682202:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829652.682214:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476980736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829652.682224:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829652.682228:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829652.682232:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829652.682237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4020 00000100:00080000:0.0:1787829652.682241:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829652.682245:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980480:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787829652.682250:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829652.682258:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829652.682272:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476980864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829652.682292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829652.682317:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829652.682350:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829653.705967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829654.730028:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829655.754050:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829655.754117:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787829655.754163:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829655.754186:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829655.754214:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829655.754228:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829655.754240:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829655.754254:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829655.754259:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829652/real 1787829655] req@ffff9a7890a3fc00 x1874671476980480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829655.754313:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476980480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829655.754326:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829655.754331:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829655.754336:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829655.754344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829652/real 1787829655] req@ffff9a7890a3da40 x1874671476980608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829655.754357:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476980608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829655.754364:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829655.754368:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829655.754377:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829655.754384:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829652/real 1787829655] req@ffff9a7890a3c3c0 x1874671476980736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829655.754397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476980736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829655.754421:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829655.754425:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829655.754429:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829655.754435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829652/real 1787829655] req@ffff9a7890a3de00 x1874671476980352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829655.754449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476980352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829655.754457:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829655.754460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829655.754463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476980352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829655.754488:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829652/real 1787829655] req@ffff9a7890a3c000 x1874671476980864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829655.754499:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476980864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829655.754508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829655.754511:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829655.754514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476980864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829656.777977:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829657.802032:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829657.802036:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829657.802048:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829657.802053:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829657.802066:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4025 00000100:00080000:2.0:1787829657.802080:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829657.802089:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829657.802136:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476980992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829657.802174:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829657.802178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829657.802180:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829657.802184:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4025 00000100:00080000:2.0:1787829657.802189:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829657.802193:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829657.802206:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829657.802221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476981120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829657.802235:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829657.802238:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476980992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829657.802239:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829657.802244:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829657.802258:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4025 00000100:00080000:2.0:1787829657.802263:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829657.802268:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829657.802275:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829657.802292:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476981248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829657.802304:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829657.802309:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829657.802311:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829657.802316:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4025 00000100:00080000:2.0:1787829657.802320:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829657.802325:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829657.802332:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829657.802333:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829657.802351:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476981376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829657.802362:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829657.802366:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829657.802369:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829657.802374:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4025 00000100:00080000:2.0:1787829657.802379:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829657.802383:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829657.802390:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829657.802399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829657/real 1787829657] req@ffff9a78841db840 x1874671476980992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829657.802406:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476981504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829657.802429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476980992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829657.802450:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829657.802456:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829657.802460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476980992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829657.802499:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829657.802524:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829657.802538:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829657/real 1787829657] req@ffff9a78841d92c0 x1874671476981120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829657.802550:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476981120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829657.802558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829657.802561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829657.802564:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829657.802570:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829657.802590:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829657.802601:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829657/real 1787829657] req@ffff9a78841d9680 x1874671476981248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829657.802613:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476981248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829657.802621:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829657.802624:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829657.802628:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829657.802634:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829657.802654:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829657.802667:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829657/real 1787829657] req@ffff9a78841d8b40 x1874671476981376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829657.802679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476981376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829657.802690:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829657.802694:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829657.802698:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829657.802719:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829657.802741:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829657.802753:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829657/real 1787829657] req@ffff9a78841da940 x1874671476981504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829657.802766:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476981504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829657.802777:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829657.802781:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829657.802784:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829658.826080:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829659.849968:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829660.873989:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829661.897993:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829662.922008:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829662.922016:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829662.922019:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829662.922028:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4030 00000100:00080000:0.0:1787829662.922038:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829662.922044:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829662.922079:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476981632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787829662.922102:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829662.922106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829662.922111:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829662.922114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829662.922120:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4030 00000100:00080000:0.0:1787829662.922123:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829662.922127:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829662.922135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829662.922145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476981760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829662.922153:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829662.922159:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829662.922162:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829662.922162:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476981632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829662.922169:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4030 00000100:00080000:0.0:1787829662.922176:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829662.922180:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829662.922183:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829662.922190:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476981888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829662.922196:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829662.922199:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829662.922201:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829662.922204:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4030 00000100:00080000:0.0:1787829662.922206:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829662.922208:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829662.922211:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829662.922220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476982016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829662.922226:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829662.922228:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829662.922229:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829662.922233:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4030 00000100:00080000:0.0:1787829662.922270:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829662.922273:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829662.922276:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829662.922287:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476982144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829662.922290:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829662.922365:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829662/real 1787829662] req@ffff9a7890a3f480 x1874671476981632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829662.922384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476981632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829662.922395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829662.922398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829662.922401:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476981632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829662.922439:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829662.922471:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829662.922481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829662/real 1787829662] req@ffff9a7890a3c000 x1874671476981760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829662.922489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476981760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829662.922495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829662.922498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829662.922500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829662.922504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829662.922516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829662.922522:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829662/real 1787829662] req@ffff9a7890a3de00 x1874671476981888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829662.922530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476981888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829662.922535:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829662.922537:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829662.922539:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476981888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829662.922543:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829662.922554:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829662.922559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829662/real 1787829662] req@ffff9a7890a3c3c0 x1874671476982016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829662.922566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476982016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829662.922572:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829662.922574:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829662.922576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829662.922579:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829662.922589:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829662.922594:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829662/real 1787829662] req@ffff9a7890a3da40 x1874671476982144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829662.922601:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476982144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829687 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829662.922607:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829662.922609:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829662.922611:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829663.946017:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829664.970075:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829665.994024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829667.018056:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829668.041953:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829668.041964:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829668.041969:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.041980:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4035 00000100:00080000:0.0:1787829668.041992:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.042000:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829668.042045:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476982272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787829668.042054:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829668.042071:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829668.042075:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829668.042076:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.042080:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4035 00000100:00080000:0.0:1787829668.042083:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.042086:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829668.042099:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829668.042111:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476982272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829668.042119:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476982400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829668.042133:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829668.042137:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829668.042140:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.042144:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4035 00000100:00080000:0.0:1787829668.042148:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.042152:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829668.042157:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829668.042167:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476982528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829668.042183:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787829668.042188:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829668.042190:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829668.042194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.042197:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4035 00000100:00080000:0.0:1787829668.042201:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.042204:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829668.042209:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829668.042217:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476982656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829668.042224:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829668.042227:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829668.042229:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.042232:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4035 00000100:00080000:0.0:1787829668.042236:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.042240:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829668.042245:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829668.042245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9a7890a3fc00 x1874671476982272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829668.042259:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476982784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829668.042266:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476982272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829668.042277:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829668.042281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829668.042284:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476982272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829668.042307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829668.042322:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829668.042330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9a7890a3da40 x1874671476982400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829668.042338:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476982400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829668.042343:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829668.042344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829668.042347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829668.042350:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829668.042360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829668.042366:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9a7890a3c3c0 x1874671476982528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829668.042373:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476982528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829668.042383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829668.042385:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829668.042387:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829668.042390:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829668.042401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829668.042406:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9a7890a3de00 x1874671476982656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829668.042413:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476982656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829668.042419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829668.042421:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829668.042423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829668.042426:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829668.042436:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829668.042441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9a7890a3c000 x1874671476982784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829668.042448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476982784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829668.042455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829668.042456:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829668.042459:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476982784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829669.066046:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829670.090013:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829671.114042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829672.138083:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829673.161908:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829673.161916:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829673.161937:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829673.161954:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4040 00000100:00080000:0.0:1787829673.161963:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829673.161970:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829673.162029:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476982912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787829673.162033:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829673.162061:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829673.162065:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829673.162068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829673.162071:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4040 00000100:00080000:0.0:1787829673.162091:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829673.162095:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829673.162105:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829673.162116:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476982912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829673.162117:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476983040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829673.162132:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829673.162135:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829673.162136:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829673.162139:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4040 00000100:00080000:0.0:1787829673.162142:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829673.162144:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829673.162147:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829673.162156:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476983168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829673.162162:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829673.162164:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829673.162165:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829673.162169:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4040 00000100:00080000:0.0:1787829673.162172:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829673.162174:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829673.162178:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829673.162180:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829673.162190:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476983296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829673.162214:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829673.162217:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829673.162218:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829673.162221:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4040 00000100:00080000:0.0:1787829673.162224:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829673.162226:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829673.162231:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829673.162241:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476983424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829673.162245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829673] req@ffff9a7890a3f480 x1874671476982912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829673.162269:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476982912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829673.162278:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829673.162281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829673.162284:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476982912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829673.162302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829673.162318:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829673.162326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829673] req@ffff9a7890a3c000 x1874671476983040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829673.162334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476983040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829673.162340:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829673.162342:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829673.162345:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829673.162348:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829673.162358:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829673.162364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829673] req@ffff9a7890a3de00 x1874671476983168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829673.162372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476983168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829673.162377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829673.162379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829673.162381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829673.162384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829673.162394:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829673.162399:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829673] req@ffff9a7890a3c3c0 x1874671476983296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829673.162407:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476983296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829673.162413:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829673.162415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829673.162417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829673.162420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829673.162430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829673.162435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829673/real 1787829673] req@ffff9a7890a3da40 x1874671476983424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829673.162442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476983424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829673.162630:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829673.162640:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829673.162646:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829674.186110:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829675.209990:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829676.233983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829677.258009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829678.281970:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829678.281984:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829678.281989:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829678.282009:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4045 00000100:00080000:0.0:1787829678.282021:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829678.282030:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829678.282046:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829678.282083:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476983552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829678.282109:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829678.282112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829678.282114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829678.282117:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4045 00000100:00080000:0.0:1787829678.282120:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829678.282123:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829678.282133:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829678.282147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476983680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829678.282155:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476983552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829678.282157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829678.282163:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829678.282167:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829678.282171:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4045 00000100:00080000:0.0:1787829678.282174:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829678.282177:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829678.282180:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829678.282189:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476983808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829678.282195:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829678.282197:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829678.282199:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829678.282202:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4045 00000100:00080000:0.0:1787829678.282205:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829678.282207:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829678.282211:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829678.282218:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476983936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829678.282224:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829678.282227:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829678.282228:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829678.282231:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4045 00000100:00080000:0.0:1787829678.282234:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829678.282238:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829678.282241:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787829678.282249:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829678.282261:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476984064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829678.282313:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9a7890a3fc00 x1874671476983552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829678.282330:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476983552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829678.282340:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829678.282343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829678.282346:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476983552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829678.282369:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829678.282394:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829678.282404:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9a7890a3da40 x1874671476983680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829678.282412:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476983680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829678.282419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829678.282421:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829678.282423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829678.282428:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829678.282440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829678.282446:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9a7890a3c3c0 x1874671476983808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829678.282454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476983808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829678.282459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829678.282461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829678.282463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829678.282467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829678.282478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829678.282483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9a7890a3de00 x1874671476983936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829678.282491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476983936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829678.282498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829678.282499:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829678.282501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476983936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829678.282505:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829678.282514:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829678.282519:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829678/real 1787829678] req@ffff9a7890a3c000 x1874671476984064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829678.282528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476984064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829678.282535:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829678.282536:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829678.282538:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829679.305928:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829680.330011:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829681.354036:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829682.377976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829683.401930:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829683.401973:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829683.401983:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829683.401988:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829683.402000:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4050 00000100:00080000:1.0:1787829683.402012:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829683.402027:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829683.402074:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476984192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829683.402099:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829683.402102:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829683.402104:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829683.402108:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4050 00000100:00080000:1.0:1787829683.402112:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829683.402117:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829683.402128:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829683.402142:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476984192:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787829683.402227:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829683.402245:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476984320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829683.402265:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829683.402269:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9a7890b1f480 x1874671476984192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829683.402270:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829683.402275:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829683.402285:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4050 00000100:00080000:1.0:1787829683.402291:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829683.402294:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476984192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829683.402298:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829683.402309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787829683.402311:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00080000:3.0:1787829683.402314:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829683.402319:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476984192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829683.402331:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476984448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829683.402341:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829683.402344:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829683.402345:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829683.402350:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4050 00000100:00100000:3.0:1787829683.402350:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984320:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829683.402355:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829683.402361:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829683.402365:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829683.402374:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476984576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829683.402386:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829683.402392:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000800:00000100:3.0:1787829683.402392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829683.402397:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829683.402401:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4050 00000100:00080000:1.0:1787829683.402404:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829683.402407:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829683.402411:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829683.402417:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9a7890b1fc00 x1874671476984320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829683.402422:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476984704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829683.402455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476984320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829683.402467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829683.402470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829683.402473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829683.402483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829683.402498:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829683.402506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9a7890b1f0c0 x1874671476984448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829683.402514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476984448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829683.402519:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829683.402521:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829683.402523:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829683.402542:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829683.402557:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829683.402564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9a7890b1f840 x1874671476984576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829683.402572:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476984576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829683.402578:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829683.402580:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829683.402583:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829683.402586:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829683.402596:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829683.402601:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829683/real 1787829683] req@ffff9a7890b1c000 x1874671476984704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829683.402608:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476984704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829683.402614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829683.402615:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829683.402617:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829684.425967:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829685.449938:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829686.473966:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829687.497955:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829688.521927:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829688.521936:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829688.521940:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829688.521949:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4056 00000100:00080000:1.0:1787829688.521958:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829688.521965:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829688.522022:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671476984832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829688.522032:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829688.522043:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829688.522063:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829688.522065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829688.522068:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4056 00000100:00080000:1.0:1787829688.522071:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829688.522075:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829688.522084:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829688.522095:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476984832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829688.522096:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671476984960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829688.522126:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829688.522129:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829688.522130:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829688.522133:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4056 00000100:00080000:1.0:1787829688.522136:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829688.522138:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829688.522142:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829688.522153:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671476985088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829688.522160:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829688.522163:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829688.522164:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829688.522166:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4056 00000100:00080000:1.0:1787829688.522176:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829688.522179:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829688.522183:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829688.522185:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829688.522195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671476985216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829688.522204:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829688.522207:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829688.522209:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829688.522214:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4056 00000100:00080000:1.0:1787829688.522219:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829688.522223:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829688.522229:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829688.522231:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829688/real 1787829688] req@ffff9a7890b1c000 x1874671476984832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829688.522262:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671476985344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829688.522267:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671476984832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829688.522294:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829688.522298:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829688.522302:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476984832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829688.522338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829688.522355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829688.522365:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829688/real 1787829688] req@ffff9a7890b1f840 x1874671476984960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829688.522374:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671476984960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829688.522393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829688.522395:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829688.522398:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476984960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829688.522403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829688.522415:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829688.522422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829688/real 1787829688] req@ffff9a7890b1f0c0 x1874671476985088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829688.522430:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671476985088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829688.522436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829688.522438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829688.522440:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829688.522444:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829688.522454:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829688.522461:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829688/real 1787829688] req@ffff9a7890b1fc00 x1874671476985216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829688.522468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671476985216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829688.522476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829688.522478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829688.522480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829688.522483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829688.522514:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829688.522527:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829688/real 1787829688] req@ffff9a7890b1f480 x1874671476985344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829688.522567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671476985344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829688.522577:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829688.522580:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829688.522583:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829689.546046:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829690.570054:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829691.594108:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829692.617939:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829693.641957:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829693.641967:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829693.641971:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829693.641981:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4061 00000100:00080000:2.0:1787829693.641991:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829693.641998:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829693.642037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829693.642037:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476985472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829693.642064:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829693.642068:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829693.642069:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829693.642073:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4061 00000100:00080000:2.0:1787829693.642075:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829693.642078:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829693.642087:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829693.642098:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476985600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829693.642111:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476985472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829693.642112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829693.642121:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829693.642124:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829693.642130:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4061 00000100:00080000:2.0:1787829693.642135:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829693.642140:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829693.642148:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829693.642164:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476985728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829693.642175:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829693.642176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829693.642182:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829693.642184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829693.642189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4061 00000100:00080000:2.0:1787829693.642195:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829693.642199:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829693.642204:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829693.642227:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476985856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829693.642235:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829693.642238:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829693.642240:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829693.642243:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4061 00000100:00080000:2.0:1787829693.642246:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829693.642250:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829693/real 1787829693] req@ffff9a78841da940 x1874671476985472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829693.642251:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829693.642258:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829693.642272:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476985984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829693.642284:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476985472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829693.642297:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829693.642301:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829693.642305:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476985472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829693.642329:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829693.642350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829693.642370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829693/real 1787829693] req@ffff9a78841d8b40 x1874671476985600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829693.642380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476985600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829693.642387:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829693.642390:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829693.642392:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829693.642397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829693.642409:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829693.642416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829693/real 1787829693] req@ffff9a78841d9680 x1874671476985728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829693.642424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476985728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829693.642430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829693.642433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829693.642435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829693.642439:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829693.642451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829693.642475:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829693/real 1787829693] req@ffff9a78841d92c0 x1874671476985856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829693.642485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476985856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829693.642493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829693.642495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829693.642498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829693.642502:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829693.642517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829693.642524:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829693/real 1787829693] req@ffff9a78841db840 x1874671476985984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829693.642533:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476985984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829693.642539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829693.642541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829693.642543:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476985984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829694.665972:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829695.690029:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829696.713948:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829697.738030:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829698.761882:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829698.761890:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829698.761894:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829698.761903:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4066 00000100:00080000:2.0:1787829698.761913:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829698.761919:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829698.761953:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476986112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829698.761973:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829698.761976:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829698.761977:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829698.761981:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4066 00000100:00080000:2.0:1787829698.761983:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829698.761987:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829698.761989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00100000:2.0:1787829698.761996:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829698.762006:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476986240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829698.762015:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829698.762017:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476986112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829698.762018:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829698.762022:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829698.762030:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4066 00000100:00080000:2.0:1787829698.762032:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829698.762034:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829698.762037:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829698.762046:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476986368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829698.762052:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829698.762055:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829698.762056:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829698.762060:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4066 00000100:00080000:2.0:1787829698.762076:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829698.762078:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829698.762082:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829698.762087:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829698.762093:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476986496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829698.762101:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829698.762104:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829698.762105:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829698.762108:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4066 00000100:00080000:2.0:1787829698.762110:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829698.762113:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829698.762115:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829698.762122:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476986624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829698.762153:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829698/real 1787829698] req@ffff9a78841db480 x1874671476986112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829698.762174:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476986112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829698.762185:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829698.762189:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829698.762193:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476986112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829698.762215:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829698.762240:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829698.762253:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829698/real 1787829698] req@ffff9a78841d83c0 x1874671476986240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829698.762266:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476986240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829698.762274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829698.762278:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829698.762281:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829698.762288:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829698.762307:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829698.762319:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829698/real 1787829698] req@ffff9a78841d8000 x1874671476986368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829698.762331:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476986368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829698.762339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829698.762344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829698.762348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829698.762355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829698.762374:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829698.762385:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829698/real 1787829698] req@ffff9a78841da580 x1874671476986496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829698.762397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476986496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829698.762407:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829698.762411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829698.762414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829698.762437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829698.762459:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829698.762471:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829698/real 1787829698] req@ffff9a78841d9e00 x1874671476986624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829698.762484:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476986624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829698.762495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829698.762498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829698.762511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829699.785996:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829700.809983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829701.833998:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829702.857985:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829703.882019:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829703.882044:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829703.882059:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829703.882068:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829703.882089:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4071 00000100:00080000:1.0:1787829703.882109:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829703.882121:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829703.882189:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476986752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829703.882245:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829703.882253:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829703.882258:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829703.882267:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4071 00000100:00080000:1.0:1787829703.882275:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829703.882282:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829703.882297:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476986752:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787829703.882298:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829703.882335:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476986880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829703.882364:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829703.882369:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829703.882371:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829703.882379:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4071 00000800:00000100:3.0:1787829703.882419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829703.882420:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829703.882433:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829703.882443:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829703.882463:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476987008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829703.882477:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829703.882485:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1787829703.882486:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829703/real 1787829703] req@ffff9a7890b1da40 x1874671476986752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829703.882490:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829703.882502:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4071 00000100:00080000:1.0:1787829703.882515:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829703.882522:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829703.882531:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829703.882542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476986752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829703.882545:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476987136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829703.882625:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829703.882631:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829703.882634:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829703.882640:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476986752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829703.882659:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829703.882664:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829703.882667:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986880:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787829703.882686:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4071 00000100:00080000:1.0:1787829703.882701:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829703.882710:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829703.882718:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829703.882728:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829703/real 1787829703] req@ffff9a7890b1d680 x1874671476986880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829703.882729:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829703.882752:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476986880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829703.882762:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476987264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829703.882764:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829703.882794:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829703.882804:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476986880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829703.882816:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829703.882861:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829703.882871:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829703/real 1787829703] req@ffff9a7890b1de00 x1874671476987008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829703.882880:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476987008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829703.882890:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829703.882893:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829703.882895:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829703.882900:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829703.882912:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829703.882918:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829703/real 1787829703] req@ffff9a7890b1e580 x1874671476987136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829703.882926:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476987136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829703.882932:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829703.882934:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829703.882935:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829703.882948:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829703.882958:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829703.882964:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829703/real 1787829703] req@ffff9a7890b1e940 x1874671476987264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829703.882972:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476987264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829703.882978:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829703.882980:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829703.882982:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829704.906072:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829705.930043:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829706.953990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829707.978010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829709.001972:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829709.001981:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829709.001985:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829709.001993:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4076 00000100:00080000:3.0:1787829709.002002:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829709.002009:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829709.002009:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829709.002044:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671476987392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829709.002067:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829709.002070:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829709.002071:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829709.002074:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4076 00000100:00080000:3.0:1787829709.002076:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829709.002079:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829709.002086:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829709.002094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671476987520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829709.002102:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829709.002105:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829709.002106:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829709.002109:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4076 00000100:00080000:3.0:1787829709.002111:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829709.002114:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829709.002117:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829709.002125:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476987392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829709.002128:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671476987648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829709.002147:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829709.002150:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829709.002151:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829709.002157:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4076 00000100:00080000:3.0:1787829709.002159:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829709.002162:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829709.002166:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829709.002175:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671476987776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829709.002181:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829709.002184:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829709.002185:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829709.002187:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4076 00000100:00080000:3.0:1787829709.002189:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829709.002191:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829709.002194:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829709.002201:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671476987904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829709.002209:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829709.002265:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9a78a6777480 x1874671476987392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829709.002285:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671476987392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829709.002298:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829709.002303:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829709.002307:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476987392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829709.002315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829709.002338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829709.002364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9a78a6774000 x1874671476987520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829709.002376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671476987520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829709.002383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829709.002385:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829709.002387:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829709.002407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829709.002422:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829709.002430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9a78a6775a40 x1874671476987648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829709.002438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671476987648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829709.002443:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829709.002446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829709.002449:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829709.002453:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829709.002463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829709.002469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9a78a67752c0 x1874671476987776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829709.002476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671476987776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829709.002482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829709.002484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829709.002486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829709.002489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829709.002499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829709.002504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829709/real 1787829709] req@ffff9a78a6774b40 x1874671476987904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829709.002511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671476987904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829709.002517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829709.002519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829709.002521:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476987904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829710.026029:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829711.049997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829712.073998:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829713.097990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829714.121924:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.121942:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829714.121947:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829714.121959:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4081 00000100:00080000:3.0:1787829714.121973:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.121982:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829714.122029:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671476988032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829714.122063:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.122067:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829714.122069:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829714.122074:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4081 00000100:00080000:3.0:1787829714.122078:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.122083:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829714.122095:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829714.122111:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671476988160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829714.122124:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.122127:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829714.122130:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829714.122133:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476988032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829714.122135:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4081 00000100:00080000:3.0:1787829714.122139:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.122142:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829714.122152:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829714.122171:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671476988288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829714.122180:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.122183:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829714.122185:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829714.122188:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4081 00000100:00080000:3.0:1787829714.122192:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.122195:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829714.122199:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829714.122207:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671476988416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829714.122215:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829714.122218:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829714.122220:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829714.122228:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4081 00000800:00000100:2.0:1787829714.122229:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787829714.122239:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829714.122243:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829714.122249:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829714.122261:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671476988544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829714.122290:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff9a78a6776580 x1874671476988032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829714.122310:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671476988032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829714.122339:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829714.122345:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829714.122353:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476988032:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829714.122357:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829714.122385:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829714.122413:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829714.122428:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff9a78a6776940 x1874671476988160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829714.122443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671476988160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829714.122453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829714.122457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829714.122460:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829714.122468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829714.122490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829714.122503:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff9a78a67770c0 x1874671476988288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829714.122521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671476988288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829714.122530:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829714.122534:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829714.122537:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829714.122546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829714.122568:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829714.122577:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff9a78a6775680 x1874671476988416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829714.122586:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671476988416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829714.122593:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829714.122595:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829714.122597:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829714.122602:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829714.122613:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829714.122619:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829714/real 1787829714] req@ffff9a78a6774780 x1874671476988544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829714.122627:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671476988544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829714.122638:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829714.122640:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829714.122642:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829715.146038:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829716.169983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829717.193978:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829718.217966:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829719.241928:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829719.241939:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829719.241945:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829719.241962:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4086 00000100:00080000:0.0:1787829719.241973:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787829719.241974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829719.241982:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829719.242020:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476988672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829719.242044:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829719.242049:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829719.242051:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829719.242055:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4086 00000100:00080000:0.0:1787829719.242059:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829719.242064:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829719.242073:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476988672:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787829719.242074:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829719.242097:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476988800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829719.242106:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829719.242109:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829719.242111:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829719.242114:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4086 00000100:00080000:0.0:1787829719.242124:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829719.242127:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829719.242131:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829719.242140:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476988928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829719.242147:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829719.242149:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829719.242151:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829719.242154:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4086 00000100:00080000:0.0:1787829719.242156:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829719.242160:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829719.242165:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829719.242173:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476989056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829719.242179:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988800:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787829719.242183:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829719.242187:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829719.242188:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829719.242192:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4086 00000100:00080000:0.0:1787829719.242196:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829719.242198:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829719.242202:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829719.242209:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787829719.242216:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476989184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829719.242234:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829719.242249:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829720.266025:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829721.290039:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829722.314342:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00010000:00010000:0.0:1787829722.314378:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829722.314406:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829722.314424:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829722.314471:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829722.314481:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829722.314492:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829722.314508:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829722.314531:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829722] req@ffff9a7890a3c3c0 x1874671476989056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829722.314568:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476989056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829722.314594:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829722.314600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829722.314605:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829722.314615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829722] req@ffff9a7890a3da40 x1874671476989184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829722.314630:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476989184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829722.314642:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829722.314646:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829722.314650:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989184:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829722.314658:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829722] req@ffff9a7890a3c000 x1874671476988800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829722.314672:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476988800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829722.314682:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829722.314685:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829722.314689:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829722.314696:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829722] req@ffff9a7890a3de00 x1874671476988928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829722.314708:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476988928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829722.314717:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829722.314721:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829722.314725:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476988928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829722.314733:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829722] req@ffff9a7890a3f480 x1874671476988672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829722.314753:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476988672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829722.314762:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829722.314766:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829722.314770:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476988672:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829723.338038:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829724.361980:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.361988:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829724.361992:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829724.362001:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4091 00000100:00080000:1.0:1787829724.362010:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.362016:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829724.362024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829724.362048:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671476989312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829724.362072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.362076:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829724.362078:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829724.362082:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4091 00000100:00080000:1.0:1787829724.362086:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.362091:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829724.362100:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829724.362116:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671476989440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829724.362127:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476989312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829724.362130:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.362140:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829724.362142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829724.362147:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4091 00000100:00080000:1.0:1787829724.362152:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.362156:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829724.362162:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829724.362176:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671476989568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829724.362187:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.362191:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829724.362196:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829724.362196:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829724.362205:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4091 00000100:00080000:1.0:1787829724.362211:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.362216:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829724.362222:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829724.362236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671476989696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829724.362261:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.362264:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829724.362266:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829724.362271:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4091 00000100:00100000:3.0:1787829724.362276:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9a7890b1e940 x1874671476989312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829724.362277:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.362287:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829724.362298:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829724.362306:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671476989312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829724.362316:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671476989824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829724.362322:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829724.362332:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829724.362338:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476989312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829724.362361:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829724.362388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829724.362401:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9a7890b1e580 x1874671476989440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829724.362413:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671476989440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829724.362421:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829724.362425:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829724.362428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829724.362434:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829724.362451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829724.362462:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9a7890b1de00 x1874671476989568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829724.362473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671476989568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829724.362481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829724.362485:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829724.362488:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829724.362493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829724.362513:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829724.362525:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9a7890b1d680 x1874671476989696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829724.362542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671476989696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829724.362552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829724.362556:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829724.362559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829724.362566:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829724.362597:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829724.362608:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9a7890b1da40 x1874671476989824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829724.362620:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671476989824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829724.362631:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829724.362634:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829724.362637:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476989824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829725.386100:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829726.410042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829727.433954:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829728.457996:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829729.481962:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829729.481973:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829729.481978:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829729.481986:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4096 00010000:00010000:0.0:1787829729.481990:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829729.482004:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829729.482010:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829729.482042:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476989952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829729.482067:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829729.482070:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829729.482071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829729.482074:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4096 00000100:00080000:2.0:1787829729.482076:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829729.482079:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829729.482087:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829729.482100:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476990080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829729.482111:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829729.482115:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829729.482117:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829729.482123:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476989952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829729.482126:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4096 00000100:00080000:2.0:1787829729.482132:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829729.482136:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829729.482142:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829729.482157:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476990208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829729.482168:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829729.482173:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829729.482175:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829729.482180:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4096 00000800:00000100:3.0:1787829729.482183:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829729.482187:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829729.482194:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829729.482200:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829729.482226:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476990336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829729.482237:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829729.482240:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829729.482242:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829729.482246:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4096 00000100:00080000:2.0:1787829729.482249:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829729.482251:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9a78841d8780 x1874671476989952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829729.482254:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829729.482263:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829729.482275:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476989952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829729.482280:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476990464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829729.482303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829729.482307:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829729.482311:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476989952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829729.482317:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829729.482335:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829729.482344:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9a78841d9a40 x1874671476990080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829729.482352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476990080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829729.482358:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829729.482360:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829729.482362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829729.482386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829729.482401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829729.482408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9a78841da1c0 x1874671476990208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829729.482415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476990208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829729.482421:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829729.482423:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829729.482425:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829729.482429:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829729.482439:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829729.482444:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9a78841db0c0 x1874671476990336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829729.482451:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476990336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829729.482457:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829729.482459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829729.482461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829729.482464:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829729.482477:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829729.482487:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9a78841dbc00 x1874671476990464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829729.482498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476990464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829729.482508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829729.482512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829729.482515:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829730.506062:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829731.529959:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829732.554008:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829733.578008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829734.601941:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829734.601951:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829734.601955:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829734.601966:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4102 00000100:00080000:2.0:1787829734.601977:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829734.601984:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829734.601991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829734.602023:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476990592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829734.602049:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829734.602053:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829734.602055:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829734.602059:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4102 00000100:00080000:2.0:1787829734.602064:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829734.602069:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829734.602079:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829734.602083:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476990592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829734.602096:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476990720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829734.602109:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829734.602113:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829734.602115:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829734.602119:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4102 00000100:00080000:2.0:1787829734.602133:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829734.602137:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829734.602142:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829734.602148:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829734.602163:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476990848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829734.602193:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829734.602195:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829734/real 1787829734] req@ffff9a78841dbc00 x1874671476990592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829734.602203:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829734.602208:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829734.602214:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4102 00000100:00100000:3.0:1787829734.602218:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476990592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829734.602221:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829734.602232:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829734.602233:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829734.602239:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829734.602244:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476990592:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829734.602245:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829734.602263:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476990976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829734.602265:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990720:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787829734.602283:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829734.602289:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000800:00000100:3.0:1787829734.602289:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829734.602295:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829734.602301:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4102 00000100:00080000:2.0:1787829734.602305:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829734.602311:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829734.602313:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829734/real 1787829734] req@ffff9a78841db0c0 x1874671476990720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787829734.602316:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829734.602331:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476991104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829734.602337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476990720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829734.602350:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829734.602355:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829734.602359:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829734.602373:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829734.602389:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829734.602398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829734/real 1787829734] req@ffff9a78841da1c0 x1874671476990848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829734.602406:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476990848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829734.602415:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829734.602418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829734.602420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829734.602436:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829734.602455:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829734.602467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829734/real 1787829734] req@ffff9a78841d9a40 x1874671476990976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829734.602479:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476990976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829734.602489:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829734.602492:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829734.602494:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476990976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829734.602498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829734.602520:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829734.602530:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829734/real 1787829734] req@ffff9a78841d8780 x1874671476991104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829734.602542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476991104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829734.602551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829734.602554:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829734.602557:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829735.625955:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829736.650024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829737.673958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829738.698027:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829739.721964:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829739.721971:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829739.721975:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829739.721984:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4107 00000100:00080000:0.0:1787829739.721992:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829739.721998:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829739.722017:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829739.722031:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476991232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829739.722052:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829739.722054:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829739.722056:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829739.722058:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4107 00000100:00080000:0.0:1787829739.722061:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829739.722064:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829739.722072:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829739.722084:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476991232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829739.722085:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476991360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829739.722098:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829739.722101:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829739.722102:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829739.722105:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4107 00000100:00080000:0.0:1787829739.722108:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829739.722110:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829739.722114:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829739.722123:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476991488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829739.722130:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829739.722145:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829739.722148:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829739.722152:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4107 00000100:00080000:0.0:1787829739.722157:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829739.722157:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829739.722164:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829739.722169:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829739.722179:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476991616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829739.722186:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829739.722189:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829739.722190:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829739.722193:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4107 00000100:00080000:0.0:1787829739.722197:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829739.722199:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829739.722203:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829739.722203:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829739/real 1787829739] req@ffff9a7890a3fc00 x1874671476991232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829739.722214:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476991744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829739.722221:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476991232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829739.722244:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829739.722247:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829739.722251:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476991232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829739.722270:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829739.722285:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829739.722293:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829739/real 1787829739] req@ffff9a7890a3f480 x1874671476991360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829739.722301:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476991360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829739.722307:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829739.722309:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829739.722311:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829739.722315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829739.722324:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829739.722330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829739/real 1787829739] req@ffff9a7890a3de00 x1874671476991488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829739.722337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476991488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829739.722342:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829739.722343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829739.722345:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829739.722349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829739.722358:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829739.722364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829739/real 1787829739] req@ffff9a7890a3c000 x1874671476991616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829739.722371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476991616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829739.722377:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829739.722379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829739.722380:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829739.722384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829739.722392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829739.722397:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829739/real 1787829739] req@ffff9a7890a3da40 x1874671476991744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829739.722404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476991744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829739.722410:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829739.722412:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829739.722414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476991744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829740.745959:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829741.769984:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829742.794022:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829743.817999:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829744.841944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829744.841953:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829744.841957:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787829744.841958:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829744.841966:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4112 00000100:00080000:3.0:1787829744.841976:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829744.841981:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829744.842013:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671476991872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829744.842035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829744.842038:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829744.842039:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829744.842042:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4112 00000100:00080000:3.0:1787829744.842044:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829744.842046:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829744.842054:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829744.842062:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671476992000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829744.842069:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829744.842071:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829744.842072:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829744.842075:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4112 00000100:00080000:3.0:1787829744.842077:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829744.842079:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829744.842082:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829744.842089:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671476992128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829744.842110:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829744.842114:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829744.842116:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829744.842123:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4112 00000100:00080000:3.0:1787829744.842128:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829744.842132:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829744.842137:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476991872:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787829744.842140:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829744.842155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671476992256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829744.842165:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829744.842168:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829744.842171:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829744.842174:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4112 00000100:00080000:3.0:1787829744.842194:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829744.842198:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829744.842202:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829744.842205:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829744.842219:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671476992384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829744.842268:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829744/real 1787829744] req@ffff9a78a67761c0 x1874671476991872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829744.842288:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671476991872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829744.842297:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829744.842301:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829744.842304:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476991872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829744.842310:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829744.842328:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829744.842338:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829744/real 1787829744] req@ffff9a78a6776d00 x1874671476992000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829744.842346:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671476992000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829744.842352:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829744.842354:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829744.842356:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829744.842360:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829744.842370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829744.842376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829744/real 1787829744] req@ffff9a78a6775e00 x1874671476992128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829744.842384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671476992128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829744.842389:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829744.842391:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829744.842392:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829744.842413:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829744.842425:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829744.842431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829744/real 1787829744] req@ffff9a78a6777840 x1874671476992256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829744.842438:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671476992256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829744.842445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829744.842446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829744.842448:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829744.842452:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829744.842461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829744.842466:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829744/real 1787829744] req@ffff9a78a6774f00 x1874671476992384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829744.842473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671476992384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829744.842479:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829744.842481:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829744.842483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829745.866022:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829746.889990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829747.914000:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829748.937979:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829749.961921:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829749.961933:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829749.961938:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829749.961950:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4117 00000100:00080000:2.0:1787829749.961962:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829749.961966:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829749.961972:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829749.962011:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476992512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829749.962037:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829749.962040:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829749.962042:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829749.962045:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4117 00000100:00080000:2.0:1787829749.962048:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829749.962052:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829749.962060:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829749.962094:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476992512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829749.962101:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476992640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829749.962111:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829749.962114:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829749.962115:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829749.962118:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4117 00000100:00080000:2.0:1787829749.962122:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829749.962124:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829749.962128:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829749.962137:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476992768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829749.962149:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829749.962155:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787829749.962156:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829749.962157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829749.962163:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4117 00000100:00080000:2.0:1787829749.962167:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829749.962170:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829749.962174:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829749.962182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476992896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829749.962188:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829749.962190:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829749.962191:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829749.962194:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4117 00000100:00080000:2.0:1787829749.962197:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829749.962200:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829749.962204:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829749.962210:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829749/real 1787829749] req@ffff9a78841d9e00 x1874671476992512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829749.962214:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476993024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829749.962239:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476992512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829749.962248:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829749.962251:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829749.962254:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476992512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829749.962275:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829749.962308:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829749.962322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829749/real 1787829749] req@ffff9a78841da580 x1874671476992640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829749.962335:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476992640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829749.962344:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829749.962348:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829749.962352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829749.962358:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829749.962388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829749.962401:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829749/real 1787829749] req@ffff9a78841d8000 x1874671476992768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829749.962412:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476992768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829749.962421:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829749.962424:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829749.962427:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829749.962434:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829749.962453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829749.962464:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829749/real 1787829749] req@ffff9a78841d83c0 x1874671476992896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829749.962476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476992896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829749.962487:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829749.962490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829749.962494:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476992896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829749.962501:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829749.962517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829749.962528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829749/real 1787829749] req@ffff9a78841db480 x1874671476993024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829749.962548:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476993024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829774 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829749.962559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829749.962562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829749.962566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829750.985924:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829752.009942:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829753.033944:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829754.057931:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829755.081930:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829755.081940:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00010000:00010000:2.0:1787829755.081943:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829755.081944:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829755.081954:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4122 00000100:00080000:0.0:1787829755.081964:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829755.081971:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829755.082006:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476993152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829755.082042:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829755.082045:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829755.082047:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829755.082051:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4122 00000100:00080000:0.0:1787829755.082054:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829755.082057:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829755.082064:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476993152:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787829755.082065:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829755.082082:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476993280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829755.082090:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829755.082092:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829755.082094:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829755.082096:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4122 00000100:00080000:0.0:1787829755.082099:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829755.082101:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829755.082105:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829755.082112:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476993408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829755.082119:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829755.082121:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829755.082125:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829755.082126:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829755.082129:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4122 00000100:00080000:0.0:1787829755.082133:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829755.082136:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829755.082140:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829755.082147:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476993536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829755.082154:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829755.082156:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829755.082157:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829755.082161:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4122 00000100:00080000:0.0:1787829755.082164:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829755.082167:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829755.082171:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829755.082175:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9a7890a3c3c0 x1874671476993152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829755.082181:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476993664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829755.082198:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476993152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829755.082207:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829755.082212:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829755.082216:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476993152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829755.082235:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829755.082252:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829755.082261:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9a7890a3da40 x1874671476993280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829755.082269:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476993280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829755.082279:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829755.082281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829755.082283:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829755.082287:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829755.082296:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829755.082302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9a7890a3c000 x1874671476993408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829755.082312:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476993408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829755.082319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829755.082321:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829755.082323:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829755.082327:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829755.082366:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829755.082377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9a7890a3de00 x1874671476993536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829755.082385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476993536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829755.082392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829755.082394:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829755.082397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829755.082417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829755.082430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829755.082438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829755/real 1787829755] req@ffff9a7890a3f480 x1874671476993664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829755.082447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476993664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829755.082454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829755.082455:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829755.082458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829756.105995:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829757.129963:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829758.154026:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829759.178071:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829760.201985:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787829760.202007:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829760.202013:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829760.202018:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.202028:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4127 00000100:00080000:2.0:1787829760.202036:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.202042:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829760.202075:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476993792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829760.202096:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829760.202099:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829760.202101:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.202103:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4127 00000100:00080000:2.0:1787829760.202106:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.202109:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829760.202117:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829760.202126:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476993920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829760.202132:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476993792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829760.202137:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829760.202140:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829760.202141:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.202145:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4127 00000100:00080000:2.0:1787829760.202147:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.202150:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829760.202153:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829760.202161:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476994048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829760.202173:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829760.202175:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829760.202177:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.202179:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4127 00000100:00080000:2.0:1787829760.202182:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.202184:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829760.202187:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829760.202191:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829760.202197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476994176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829760.202205:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829760.202209:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829760.202211:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829760.202216:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4127 00000100:00080000:2.0:1787829760.202221:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829760.202225:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829760.202230:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829760.202232:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9a78841db840 x1874671476993792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829760.202246:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476994304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829760.202250:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476993792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829760.202278:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829760.202283:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829760.202287:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476993792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829760.202306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829760.202321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829760.202331:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9a78841d92c0 x1874671476993920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829760.202339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476993920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829760.202345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829760.202347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829760.202349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476993920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829760.202353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829760.202362:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829760.202368:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9a78841d9680 x1874671476994048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829760.202375:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476994048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829760.202380:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829760.202382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829760.202384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829760.202387:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829760.202397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829760.202402:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9a78841d8b40 x1874671476994176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829760.202409:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476994176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829760.202416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829760.202417:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829760.202419:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829760.202422:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829760.202431:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829760.202437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829760/real 1787829760] req@ffff9a78841da940 x1874671476994304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829760.202448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476994304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829760.202458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829760.202461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829760.202464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829761.226011:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829762.250031:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829763.274009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829764.298035:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829765.321924:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829765.321935:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829765.321941:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829765.321952:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4132 00000100:00080000:2.0:1787829765.321976:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829765.321984:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829765.322012:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829765.322035:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671476994432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829765.322063:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829765.322066:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829765.322068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829765.322072:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4132 00000100:00080000:2.0:1787829765.322076:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829765.322081:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829765.322091:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829765.322102:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476994432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829765.322107:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671476994560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829765.322120:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829765.322124:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829765.322126:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829765.322131:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4132 00000100:00080000:2.0:1787829765.322135:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829765.322139:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829765.322144:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829765.322157:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671476994688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829765.322159:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829765.322179:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829765.322183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829765.322185:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829765.322190:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4132 00000100:00080000:2.0:1787829765.322193:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829765.322198:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829765.322204:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829765.322209:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9a78841da940 x1874671476994432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829765.322220:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671476994816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829765.322230:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671476994432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829765.322232:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829765.322240:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829765.322243:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829765.322251:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4132 00000100:00080000:3.0:1787829765.322255:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829765.322256:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829765.322263:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829765.322265:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:2.0:1787829765.322269:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829765.322272:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476994432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829765.322283:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671476994944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829765.322300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829765.322321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829765.322330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9a78841d8b40 x1874671476994560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829765.322339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671476994560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829765.322345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829765.322347:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829765.322349:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829765.322353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829765.322369:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829765.322380:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9a78841d9680 x1874671476994688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829765.322392:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671476994688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829765.322401:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829765.322405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829765.322408:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829765.322414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829765.322430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829765.322438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9a78841d92c0 x1874671476994816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829765.322446:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671476994816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829765.322453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829765.322454:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829765.322457:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829765.322460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829765.322470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829765.322475:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829765/real 1787829765] req@ffff9a78841db840 x1874671476994944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829765.322482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671476994944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829765.322488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829765.322490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829765.322492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476994944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829766.346045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829767.370003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829768.393987:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829769.418020:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829770.441955:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829770.441964:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829770.441968:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829770.441977:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4137 00000100:00080000:2.0:1787829770.441988:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829770.441996:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829770.442008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829770.442038:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671476995072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829770.442059:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829770.442062:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829770.442063:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829770.442066:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4137 00000100:00080000:2.0:1787829770.442069:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829770.442072:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829770.442080:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829770.442090:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671476995200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829770.442098:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829770.442100:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829770.442102:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829770.442105:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4137 00000100:00080000:2.0:1787829770.442107:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829770.442110:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829770.442113:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829770.442122:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476995072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829770.442123:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671476995328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829770.442143:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829770.442146:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829770.442148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829770.442151:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4137 00000100:00080000:2.0:1787829770.442153:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829770.442155:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829770.442159:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829770.442167:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671476995456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829770.442173:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829770.442175:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829770.442177:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829770.442179:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4137 00000100:00080000:2.0:1787829770.442182:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829770.442184:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829770.442186:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787829770.442194:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829770.442195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671476995584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829770.442242:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff9a78841db480 x1874671476995072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829770.442277:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671476995072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829770.442290:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829770.442294:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829770.442299:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476995072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829770.442308:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829770.442335:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829770.442350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff9a78841d83c0 x1874671476995200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829770.442364:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671476995200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829770.442374:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829770.442378:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829770.442380:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829770.442401:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829770.442418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829770.442427:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff9a78841d8000 x1874671476995328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829770.442435:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671476995328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829770.442441:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829770.442443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829770.442446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829770.442451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829770.442461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829770.442468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff9a78841da580 x1874671476995456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829770.442475:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671476995456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829770.442482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829770.442483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829770.442485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829770.442489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829770.442499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829770.442504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829770/real 1787829770] req@ffff9a78841d9e00 x1874671476995584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829770.442512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671476995584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829770.442518:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829770.442520:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829770.442521:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829771.466069:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829772.490043:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829773.514046:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829774.537989:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829775.561958:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829775.561969:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829775.561973:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829775.561982:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4143 00000100:00080000:0.0:1787829775.561992:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829775.561999:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787829775.562042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829775.562049:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476995712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829775.562070:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829775.562073:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829775.562075:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829775.562078:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4143 00000100:00080000:0.0:1787829775.562080:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829775.562083:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829775.562093:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829775.562103:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476995840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829775.562112:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476995712:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829775.562113:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829775.562120:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829775.562122:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829775.562126:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4143 00000100:00080000:0.0:1787829775.562128:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829775.562130:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829775.562132:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829775.562144:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476995968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829775.562166:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829775.562174:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829775.562176:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829775.562180:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4143 00000100:00080000:0.0:1787829775.562184:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829775.562189:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829775.562192:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829775.562198:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829775.562209:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476996096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829775.562216:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829775.562218:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829775.562219:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829775.562222:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4143 00000100:00080000:0.0:1787829775.562225:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829775.562228:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829775.562232:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829775.562242:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476996224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829775.562248:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829775/real 1787829775] req@ffff9a7890a3fc00 x1874671476995712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829775.562269:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476995712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829775.562281:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829775.562285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829775.562289:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476995712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829775.562313:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829775.562332:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829775.562350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829775/real 1787829775] req@ffff9a7890a3f480 x1874671476995840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829775.562363:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476995840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829775.562371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829775.562373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829775.562377:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829775.562382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829775.562397:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829775.562406:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829775/real 1787829775] req@ffff9a7890a3de00 x1874671476995968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829775.562416:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476995968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829775.562435:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829775.562438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829775.562441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476995968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829775.562447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829775.562463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829775.562471:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829775/real 1787829775] req@ffff9a7890a3c000 x1874671476996096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829775.562482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476996096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829775.562491:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829775.562494:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829775.562497:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829775.562502:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829775.562516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829775.562524:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829775/real 1787829775] req@ffff9a7890a3da40 x1874671476996224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829775.562534:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476996224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829775.562542:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829775.562545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829775.562549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829776.586018:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829777.609978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829778.633998:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829779.658011:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829780.681998:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829780.682010:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829780.682014:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829780.682025:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4148 00000100:00080000:0.0:1787829780.682037:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829780.682044:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829780.682065:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829780.682087:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476996352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829780.682112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829780.682114:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829780.682116:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829780.682119:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4148 00000100:00080000:0.0:1787829780.682121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829780.682124:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829780.682136:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829780.682147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476996480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829780.682154:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829780.682158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829780.682159:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476996352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829780.682160:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829780.682172:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4148 00000100:00080000:0.0:1787829780.682178:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829780.682181:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829780.682183:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829780.682195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476996608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829780.682202:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829780.682204:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829780.682206:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829780.682209:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4148 00000100:00080000:0.0:1787829780.682210:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829780.682213:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829780.682218:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829780.682226:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476996736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829780.682232:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829780.682235:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829780.682236:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829780.682239:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4148 00000100:00080000:0.0:1787829780.682245:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829780.682246:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829780.682250:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829780.682254:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829780.682261:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476996864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829780.682302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829780/real 1787829780] req@ffff9a7890a3c3c0 x1874671476996352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829780.682319:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476996352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829780.682329:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829780.682333:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829780.682336:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476996352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829780.682357:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829780.682377:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829780.682388:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829780/real 1787829780] req@ffff9a7890a3da40 x1874671476996480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829780.682398:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476996480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829780.682405:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829780.682408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829780.682411:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829780.682417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829780.682432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829780.682441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829780/real 1787829780] req@ffff9a7890a3c000 x1874671476996608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829780.682450:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476996608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829780.682458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829780.682460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829780.682463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829780.682468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829780.682483:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829780.682492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829780/real 1787829780] req@ffff9a7890a3de00 x1874671476996736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829780.682501:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476996736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829780.682510:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829780.682513:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829780.682516:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829780.682521:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829780.682535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829780.682543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829780/real 1787829780] req@ffff9a7890a3f480 x1874671476996864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829780.682566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476996864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829780.682575:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829780.682578:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829780.682581:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476996864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829781.706051:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829782.730040:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829783.754016:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829784.778024:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829785.801974:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829785.801986:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829785.801992:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829785.802004:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4153 00000100:00080000:0.0:1787829785.802016:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829785.802024:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829785.802066:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829785.802067:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476996992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829785.802093:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829785.802098:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829785.802101:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829785.802105:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4153 00000100:00080000:0.0:1787829785.802109:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829785.802113:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829785.802125:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829785.802133:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476996992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829785.802140:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476997120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829785.802158:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829785.802162:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829785.802164:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829785.802168:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4153 00000100:00080000:0.0:1787829785.802172:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829785.802175:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829785.802180:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829785.802195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476997248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829785.802205:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829785.802209:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829785.802210:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829785.802214:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4153 00000100:00080000:0.0:1787829785.802218:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829785.802222:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829785.802227:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829785.802237:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476997376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829785.802253:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829785.802256:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829785.802258:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829785.802265:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997120:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787829785.802266:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4153 00000100:00080000:0.0:1787829785.802270:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829785.802274:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829785.802280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829785.802293:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476997504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829785.802312:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829785.802334:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829785.802355:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829786.825942:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829787.849959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829788.874008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829788.874068:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787829788.874101:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829788.874117:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829788.874153:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829788.874164:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829788.874173:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829788.874183:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829788.874207:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829785/real 1787829788] req@ffff9a7890a3da40 x1874671476997504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829788.874233:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476997504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829788.874246:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829788.874251:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829788.874256:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997504:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829788.874265:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829785/real 1787829788] req@ffff9a7890a3f480 x1874671476997120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829788.874276:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476997120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829788.874290:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829788.874294:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829788.874297:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829788.874304:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829785/real 1787829788] req@ffff9a7890a3de00 x1874671476997248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829788.874315:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476997248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829788.874323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829788.874327:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829788.874331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829788.874337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829785/real 1787829788] req@ffff9a7890a3c000 x1874671476997376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829788.874349:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476997376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829788.874357:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829788.874361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829788.874364:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829788.874369:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829785/real 1787829788] req@ffff9a7890a3fc00 x1874671476996992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829788.874380:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476996992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829788.874387:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829788.874390:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829788.874393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476996992:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787829789.897953:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829790.921933:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829790.921944:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829790.921948:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829790.921959:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4158 00000100:00080000:0.0:1787829790.921969:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829790.921975:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787829790.922003:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829790.922005:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476997632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829790.922028:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829790.922031:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829790.922032:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829790.922035:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4158 00000100:00080000:0.0:1787829790.922037:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829790.922040:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829790.922049:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829790.922058:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476997760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829790.922067:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829790.922069:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476997632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829790.922070:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829790.922073:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829790.922079:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4158 00000100:00080000:0.0:1787829790.922082:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829790.922084:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829790.922087:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829790.922100:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476997888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829790.922111:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829790.922115:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829790.922117:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829790.922121:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4158 00000100:00080000:0.0:1787829790.922136:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829790.922143:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829790.922144:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787829790.922154:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829790.922169:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476998016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829790.922186:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829790.922190:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829790.922192:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829790.922199:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4158 00000100:00100000:3.0:1787829790.922200:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829790/real 1787829790] req@ffff9a7890a3c3c0 x1874671476997632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829790.922204:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829790.922214:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829790.922221:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829790.922239:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476998144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829790.922241:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476997632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829790.922274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829790.922280:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829790.922285:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476997632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829790.922311:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829790.922339:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829790.922371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829790/real 1787829790] req@ffff9a7890a3fc00 x1874671476997760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829790.922389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476997760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829790.922399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829790.922403:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829790.922406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829790.922414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829790.922438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829790.922451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829790/real 1787829790] req@ffff9a7890a3c000 x1874671476997888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829790.922465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476997888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829790.922474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829790.922477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829790.922482:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476997888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829790.922489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829790.922511:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829790.922524:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829790/real 1787829790] req@ffff9a7890a3de00 x1874671476998016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829790.922538:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476998016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829790.922549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829790.922553:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829790.922557:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829790.922564:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829790.922585:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829790.922597:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829790/real 1787829790] req@ffff9a7890a3f480 x1874671476998144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829790.922611:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476998144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829790.922622:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829790.922626:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829790.922630:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829791.946000:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829792.970003:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829793.993950:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829795.018070:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829796.041897:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829796.041907:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829796.041911:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829796.041922:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4163 00000100:00080000:0.0:1787829796.041933:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829796.041941:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829796.041944:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829796.041979:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671476998272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829796.042005:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829796.042008:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829796.042009:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829796.042012:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4163 00000100:00080000:0.0:1787829796.042016:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829796.042020:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829796.042029:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829796.042036:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476998272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829796.042048:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671476998400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829796.042068:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829796.042072:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829796.042074:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829796.042078:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4163 00000100:00080000:0.0:1787829796.042083:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829796.042089:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829796.042096:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829796.042100:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829796.042112:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476998528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829796.042124:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829796.042128:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829796.042130:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829796.042134:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4163 00000100:00080000:0.0:1787829796.042144:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829796.042144:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9a7890a3da40 x1874671476998272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829796.042152:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829796.042161:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829796.042168:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671476998272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829796.042174:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476998656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829796.042194:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829796.042195:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829796.042197:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829796.042202:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829796.042206:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787829796.042207:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4163 00000100:00080000:0.0:1787829796.042213:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829796.042217:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476998272:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829796.042219:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829796.042224:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829796.042232:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476998784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829796.042246:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829796.042273:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829796.042287:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9a7890a3f480 x1874671476998400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829796.042301:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671476998400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829796.042313:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829796.042316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829796.042318:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829796.042322:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829796.042337:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829796.042345:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9a7890a3de00 x1874671476998528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829796.042352:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476998528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829796.042357:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829796.042359:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829796.042361:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829796.042365:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829796.042375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829796.042381:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9a7890a3c000 x1874671476998656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829796.042389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476998656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829796.042394:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829796.042396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829796.042399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829796.042403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829796.042412:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829796.042417:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829796/real 1787829796] req@ffff9a7890a3fc00 x1874671476998784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829796.042424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476998784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829796.042430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829796.042432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829796.042435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476998784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829797.066068:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829798.090002:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829799.114050:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829800.138023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829801.161949:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.161964:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829801.161970:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.161985:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4168 00010000:00010000:0.0:1787829801.161995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829801.162000:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829801.162011:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829801.162062:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671476998912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829801.162093:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.162097:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829801.162099:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.162104:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4168 00000100:00080000:2.0:1787829801.162107:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829801.162113:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829801.162125:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829801.162139:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476998912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829801.162142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671476999040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829801.162159:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.162162:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829801.162164:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.162169:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4168 00000100:00080000:2.0:1787829801.162174:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829801.162178:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829801.162184:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829801.162198:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671476999168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829801.162209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.162213:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829801.162215:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.162219:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4168 00000100:00080000:2.0:1787829801.162224:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829801.162229:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829801.162232:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829801.162240:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829801.162253:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671476999296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829801.162262:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829801.162265:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829801.162268:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829801.162272:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4168 00000100:00080000:2.0:1787829801.162276:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829801.162280:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829801.162285:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829801.162298:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671476999424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829801.162308:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9a78841d8780 x1874671476998912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829801.162340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671476998912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829801.162355:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829801.162361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829801.162365:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476998912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829801.162392:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829801.162419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829801.162435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9a78841d9a40 x1874671476999040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829801.162450:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671476999040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829801.162459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829801.162463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829801.162467:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829801.162473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829801.162495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829801.162508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9a78841da1c0 x1874671476999168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829801.162521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671476999168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829801.162529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829801.162532:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829801.162536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829801.162542:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829801.162561:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829801.162573:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9a78841db0c0 x1874671476999296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829801.162586:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671476999296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829801.162597:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829801.162600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829801.162603:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829801.162610:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829801.162630:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829801.162642:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829801/real 1787829801] req@ffff9a78841dbc00 x1874671476999424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829801.162656:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671476999424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829801.162667:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829801.162671:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829801.162674:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829802.186014:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829803.210040:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829804.234085:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829805.258063:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829806.281987:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829806.281999:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829806.282004:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829806.282015:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4173 00010000:00010000:1.0:1787829806.282024:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829806.282028:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829806.282036:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829806.282085:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671476999552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829806.282116:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829806.282121:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829806.282123:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829806.282127:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4173 00000100:00080000:0.0:1787829806.282133:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829806.282139:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829806.282156:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476999552:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787829806.282161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829806.282179:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671476999680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829806.282196:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829806.282200:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829806.282202:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829806.282207:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4173 00000100:00080000:0.0:1787829806.282220:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829806.282228:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829806.282235:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829806.282242:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829806.282257:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671476999808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829806.282269:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829806.282273:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829806.282276:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829806.282281:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829806] req@ffff9a7890a3c3c0 x1874671476999552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829806.282283:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4173 00000100:00080000:0.0:1787829806.282292:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829806.282299:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829806.282304:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671476999552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787829806.282308:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787829806.282319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829806.282325:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787829806.282327:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671476999936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829806.282331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671476999552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829806.282345:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829806.282347:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829806.282349:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829806.282353:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4173 00000100:00080000:0.0:1787829806.282356:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829806.282360:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829806.282364:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829806.282364:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787829806.282377:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477000064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829806.282387:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829806.282401:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829806] req@ffff9a7890a3fc00 x1874671476999680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829806.282412:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671476999680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829806.282418:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829806.282420:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829806.282423:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829806.282427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829806.282437:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829806.282442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829806] req@ffff9a7890a3c000 x1874671476999808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829806.282450:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671476999808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829806.282455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829806.282457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829806.282458:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829806.282462:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829806.282472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829806.282478:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829806] req@ffff9a7890a3de00 x1874671476999936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829806.282485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671476999936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829806.282493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829806.282495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829806.282496:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671476999936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829806.282506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829806.282515:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829806.282521:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829806/real 1787829806] req@ffff9a7890a3f480 x1874671477000064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829806.282528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477000064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829806.282535:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829806.282536:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829806.282538:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829807.306074:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829808.330010:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829809.354083:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829810.378019:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829811.401927:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829811.401938:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829811.401943:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829811.401955:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4178 00000100:00080000:0.0:1787829811.401966:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829811.401973:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829811.401978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829811.402019:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477000192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829811.402042:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829811.402046:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829811.402047:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829811.402050:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4178 00000100:00080000:0.0:1787829811.402053:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829811.402056:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829811.402067:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829811.402077:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477000192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829811.402086:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477000320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829811.402106:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829811.402110:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829811.402113:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829811.402118:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4178 00000100:00080000:0.0:1787829811.402136:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829811.402140:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829811.402144:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829811.402149:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829811.402159:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477000448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829811.402168:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829811.402171:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829811.402173:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829811.402176:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4178 00000100:00080000:0.0:1787829811.402178:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829811.402181:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829811.402185:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829811.402194:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477000576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829811.402201:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829811.402204:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9a7890a3da40 x1874671477000192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829811.402205:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829811.402208:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829811.402215:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4178 00000100:00080000:0.0:1787829811.402221:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829811.402227:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829811.402230:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829811.402242:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477000704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829811.402244:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477000192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829811.402264:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829811.402269:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829811.402274:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477000192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829811.402300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829811.402322:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829811.402336:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9a7890a3f480 x1874671477000320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829811.402351:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477000320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829811.402361:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829811.402364:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829811.402367:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829811.402371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829811.402388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829811.402396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9a7890a3de00 x1874671477000448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829811.402403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477000448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829811.402409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829811.402412:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829811.402414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829811.402418:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829811.402430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829811.402436:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9a7890a3c000 x1874671477000576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829811.402449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477000576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829811.402456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829811.402458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829811.402462:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829811.402468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829811.402488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829811.402500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829811/real 1787829811] req@ffff9a7890a3fc00 x1874671477000704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829811.402513:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477000704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829811.402521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829811.402523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829811.402526:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829812.426059:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829813.450009:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829814.474063:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829815.497938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829816.521934:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829816.522004:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829816.522017:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829816.522023:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829816.522037:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4184 00000100:00080000:2.0:1787829816.522052:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829816.522061:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829816.522111:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477000832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829816.522146:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829816.522150:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829816.522153:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829816.522158:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4184 00000100:00080000:2.0:1787829816.522163:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829816.522170:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829816.522184:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829816.522195:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477000832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829816.522205:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477000960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829816.522222:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829816.522227:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829816.522231:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829816.522237:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4184 00000100:00080000:2.0:1787829816.522261:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829816.522267:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829816.522275:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829816.522279:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829816.522301:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477001088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829816.522313:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829816.522317:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829816.522320:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829816.522326:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4184 00000100:00080000:2.0:1787829816.522331:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829816.522333:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829816/real 1787829816] req@ffff9a78841dbc00 x1874671477000832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829816.522339:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829816.522352:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829816.522365:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477000832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829816.522372:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477001216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829816.522387:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829816.522394:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829816.522402:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829816.522405:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829816.522407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787829816.522412:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4184 00000100:00100000:3.0:1787829816.522419:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477000832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829816.522420:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829816.522432:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829816.522439:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829816.522459:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477001344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829816.522462:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829816.522510:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829816.522530:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829816/real 1787829816] req@ffff9a78841db0c0 x1874671477000960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829816.522540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477000960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829816.522547:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829816.522550:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829816.522552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477000960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829816.522557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829816.522576:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829816.522590:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829816/real 1787829816] req@ffff9a78841da1c0 x1874671477001088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829816.522620:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477001088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829816.522635:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829816.522639:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829816.522643:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829816.522650:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829816.522672:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829816.522682:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829816/real 1787829816] req@ffff9a78841d9a40 x1874671477001216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829816.522690:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477001216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829816.522697:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829816.522699:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829816.522709:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829816.522726:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829816.522739:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829816.522746:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829816/real 1787829816] req@ffff9a78841d8780 x1874671477001344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829816.522759:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477001344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829816.522765:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829816.522768:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829816.522770:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829817.545992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829818.569980:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829819.593959:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829820.617986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829821.641999:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829821.642011:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829821.642016:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829821.642028:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4189 00000100:00080000:0.0:1787829821.642041:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829821.642048:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787829821.642054:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829821.642093:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477001472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829821.642117:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829821.642120:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829821.642121:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829821.642124:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4189 00000100:00080000:0.0:1787829821.642127:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829821.642129:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829821.642139:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829821.642151:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477001600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829821.642160:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829821.642163:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829821.642165:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829821.642171:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477001472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829821.642172:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4189 00000100:00080000:0.0:1787829821.642176:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829821.642184:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829821.642189:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829821.642200:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477001728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829821.642211:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829821.642216:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829821.642219:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829821.642224:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4189 00000100:00080000:0.0:1787829821.642241:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829821.642246:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829821.642254:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829821.642262:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829821.642268:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477001856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829821.642278:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829821.642281:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829821.642283:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829821.642287:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4189 00000100:00080000:0.0:1787829821.642292:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829821.642295:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829821.642298:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829821.642307:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477001984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829821.642337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829821/real 1787829821] req@ffff9a7890a3c3c0 x1874671477001472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829821.642357:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477001472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829821.642369:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829821.642374:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829821.642379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477001472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829821.642405:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829821.642432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829821.642446:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829821/real 1787829821] req@ffff9a7890a3fc00 x1874671477001600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829821.642459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477001600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829821.642465:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829821.642468:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829821.642470:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829821.642475:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829821.642490:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829821.642498:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829821/real 1787829821] req@ffff9a7890a3c000 x1874671477001728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829821.642506:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477001728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829821.642511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829821.642513:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829821.642515:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829821.642519:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829821.642530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829821.642536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829821/real 1787829821] req@ffff9a7890a3de00 x1874671477001856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829821.642543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477001856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829821.642549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829821.642551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829821.642553:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829821.642557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829821.642584:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829821.642593:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829821/real 1787829821] req@ffff9a7890a3f480 x1874671477001984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829821.642601:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477001984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829821.642608:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829821.642610:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829821.642612:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477001984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829822.666013:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829823.690005:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829824.714060:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829825.738002:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829826.761922:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829826.761931:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829826.761936:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829826.761945:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4194 00000100:00080000:1.0:1787829826.761955:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829826.761962:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829826.761995:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477002112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829826.762021:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829826.762024:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829826.762025:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829826.762028:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4194 00000100:00080000:1.0:1787829826.762030:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829826.762035:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829826.762044:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787829826.762050:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829826.762058:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477002240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829826.762070:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829826.762072:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477002112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829826.762076:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829826.762080:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829826.762085:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4194 00000100:00080000:1.0:1787829826.762089:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829826.762092:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829826.762097:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829826.762106:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477002368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829826.762112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829826.762114:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829826.762116:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829826.762119:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4194 00000100:00080000:1.0:1787829826.762121:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829826.762123:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:2.0:1787829826.762127:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829826.762130:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829826.762139:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477002496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829826.762145:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829826.762147:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829826.762148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829826.762151:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4194 00000100:00080000:1.0:1787829826.762153:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829826.762156:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829826.762159:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829826.762166:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829826/real 1787829826] req@ffff9a7890b1f480 x1874671477002112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829826.762169:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477002624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829826.762196:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477002112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829826.762204:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829826.762207:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829826.762209:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477002112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829826.762224:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829826.762237:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829826.762244:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829826/real 1787829826] req@ffff9a7890b1fc00 x1874671477002240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829826.762251:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477002240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829826.762256:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829826.762258:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829826.762260:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829826.762264:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829826.762272:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829826.762277:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829826/real 1787829826] req@ffff9a7890b1f0c0 x1874671477002368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829826.762284:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477002368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829826.762288:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829826.762289:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829826.762291:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829826.762294:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829826.762302:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829826.762307:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829826/real 1787829826] req@ffff9a7890b1f840 x1874671477002496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829826.762313:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477002496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829826.762319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829826.762320:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829826.762322:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829826.762324:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829826.762332:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829826.762337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829826/real 1787829826] req@ffff9a7890b1c000 x1874671477002624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829826.762344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477002624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829826.762349:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829826.762351:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829826.762352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829827.786068:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829828.809993:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829829.833996:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829830.857952:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829831.881947:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829831.881957:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829831.881961:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787829831.881971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829831.881972:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4199 00000100:00080000:1.0:1787829831.881982:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829831.881991:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829831.882026:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477002752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829831.882062:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829831.882066:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829831.882068:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829831.882071:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4199 00000100:00080000:1.0:1787829831.882075:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829831.882078:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829831.882087:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829831.882101:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477002880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829831.882112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829831.882115:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829831.882117:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829831.882120:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4199 00000100:00080000:1.0:1787829831.882124:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829831.882127:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829831.882131:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829831.882145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477003008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829831.882157:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477002752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829831.882161:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829831.882172:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829831.882174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829831.882179:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4199 00000100:00080000:1.0:1787829831.882182:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829831.882186:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829831.882192:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829831.882205:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477003136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829831.882215:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829831.882218:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829831.882220:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829831.882224:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4199 00000100:00080000:1.0:1787829831.882251:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829831.882255:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829831.882260:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829831.882268:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829831.882279:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477003264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829831.882349:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829831/real 1787829831] req@ffff9a7890b1c000 x1874671477002752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829831.882372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477002752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829831.882384:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829831.882389:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829831.882393:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477002752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829831.882402:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829831.882428:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829831.882442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829831/real 1787829831] req@ffff9a7890b1f840 x1874671477002880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829831.882454:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477002880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829831.882468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829831.882471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829831.882475:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477002880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829831.882520:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829831.882542:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829831.882554:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829831/real 1787829831] req@ffff9a7890b1f0c0 x1874671477003008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829831.882566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477003008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829831.882573:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829831.882576:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829831.882579:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829831.882585:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829831.882602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829831.882611:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829831/real 1787829831] req@ffff9a7890b1fc00 x1874671477003136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829831.882621:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477003136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829831.882631:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829831.882634:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829831.882637:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829831.882643:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829831.882661:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829831.882671:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829831/real 1787829831] req@ffff9a7890b1f480 x1874671477003264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829831.882682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477003264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829831.882691:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829831.882694:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829831.882697:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829832.905942:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829833.929992:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829834.953987:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829835.977969:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829837.001960:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829837.001974:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829837.001978:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829837.001990:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4204 00000100:00080000:0.0:1787829837.002003:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829837.002013:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787829837.002013:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829837.002058:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477003392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829837.002091:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829837.002095:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829837.002097:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829837.002101:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4204 00000100:00080000:0.0:1787829837.002105:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829837.002109:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829837.002120:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829837.002136:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477003520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829837.002141:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477003392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829837.002153:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829837.002156:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829837.002158:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829837.002163:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4204 00000100:00080000:0.0:1787829837.002167:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829837.002171:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829837.002176:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829837.002188:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477003648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829837.002198:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829837.002201:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829837.002203:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1787829837.002208:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829837.002211:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4204 00000100:00080000:0.0:1787829837.002216:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829837.002220:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829837.002225:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829837.002237:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477003776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829837.002252:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829837.002256:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9a7890a3da40 x1874671477003392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829837.002257:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829837.002261:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829837.002270:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4204 00000100:00080000:0.0:1787829837.002274:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829837.002278:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829837.002284:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829837.002288:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477003392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829837.002296:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477003904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829837.002298:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.002306:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829837.002314:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477003392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829837.002337:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829837.002352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829837.002360:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9a7890a3f480 x1874671477003520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829837.002367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477003520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829837.002372:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.002374:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829837.002376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829837.002379:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829837.002388:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829837.002393:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9a7890a3de00 x1874671477003648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829837.002399:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477003648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829837.002403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.002405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829837.002406:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829837.002409:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829837.002418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829837.002423:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9a7890a3c000 x1874671477003776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829837.002429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477003776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829837.002436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.002437:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829837.002439:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829837.002442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829837.002449:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829837.002454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829837/real 1787829837] req@ffff9a7890a3fc00 x1874671477003904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829837.002474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477003904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829837.002481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829837.002483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829837.002485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477003904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829838.026036:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829839.050025:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829840.074016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829841.098037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829842.121950:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829842.121958:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829842.121961:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.121970:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4209 00000100:00080000:1.0:1787829842.121978:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.121984:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829842.122004:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829842.122016:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477004032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829842.122037:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829842.122040:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829842.122049:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.122051:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4209 00000100:00080000:1.0:1787829842.122054:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.122056:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829842.122064:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829842.122074:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477004160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829842.122081:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829842.122084:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829842.122085:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.122087:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4209 00000100:00080000:1.0:1787829842.122089:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.122091:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829842.122095:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829842.122106:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477004288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829842.122115:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477004032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829842.122124:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829842.122129:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829842.122132:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.122137:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4209 00000100:00080000:1.0:1787829842.122141:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.122146:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829842.122153:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829842.122172:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477004416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829842.122180:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829842.122183:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829842.122184:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829842.122187:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4209 00000100:00080000:1.0:1787829842.122196:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829842.122199:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829842.122202:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:2.0:1787829842.122208:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829842.122213:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477004544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829842.122291:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9a7890b1da40 x1874671477004032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829842.122313:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477004032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829842.122325:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829842.122330:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829842.122335:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477004032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829842.122343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829842.122367:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829842.122380:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9a7890b1d680 x1874671477004160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829842.122391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477004160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829842.122398:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829842.122401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829842.122405:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829842.122447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829842.122466:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829842.122477:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9a7890b1de00 x1874671477004288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829842.122488:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477004288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829842.122495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829842.122498:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829842.122501:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829842.122507:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829842.122526:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829842.122537:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9a7890b1e580 x1874671477004416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829842.122548:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477004416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829842.122558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829842.122561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829842.122564:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829842.122571:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829842.122590:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829842.122601:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829842/real 1787829842] req@ffff9a7890b1e940 x1874671477004544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829842.122613:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477004544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829842.122622:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829842.122625:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829842.122628:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829843.146025:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829844.170024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829845.194052:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829846.218010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829847.241974:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.241988:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829847.241994:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829847.242007:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4214 00000100:00080000:1.0:1787829847.242021:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.242030:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829847.242048:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829847.242081:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477004672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829847.242116:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.242120:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829847.242122:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829847.242126:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4214 00000100:00080000:1.0:1787829847.242130:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.242134:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829847.242146:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829847.242160:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477004800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829847.242164:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477004672:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829847.242179:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.242183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829847.242185:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829847.242190:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4214 00000100:00080000:1.0:1787829847.242194:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.242198:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829847.242203:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829847.242216:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477004928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829847.242232:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.242238:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:2.0:1787829847.242238:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829847.242243:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829847.242249:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4214 00000100:00080000:1.0:1787829847.242253:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.242257:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829847.242263:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829847.242275:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477005056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829847.242283:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829847.242287:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829847.242289:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829847.242294:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9a7890b1e940 x1874671477004672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829847.242296:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4214 00000100:00080000:1.0:1787829847.242303:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829847.242309:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829847.242314:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829847.242315:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477004672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829847.242326:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787829847.242330:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477005184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829847.242331:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829847.242337:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477004672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829847.242365:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829847.242381:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829847.242402:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9a7890b1e580 x1874671477004800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829847.242411:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477004800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829847.242416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829847.242418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829847.242420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829847.242423:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829847.242436:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829847.242442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9a7890b1de00 x1874671477004928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829847.242450:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477004928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829847.242457:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829847.242460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829847.242464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477004928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829847.242469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829847.242487:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829847.242498:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9a7890b1d680 x1874671477005056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829847.242508:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477005056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829847.242518:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829847.242521:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829847.242524:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829847.242529:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829847.242546:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829847.242556:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829847/real 1787829847] req@ffff9a7890b1da40 x1874671477005184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829847.242567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477005184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829847.242576:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829847.242579:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829847.242583:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829848.266009:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829849.290006:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829850.313991:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829851.338019:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829852.361909:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829852.361917:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829852.361921:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829852.361931:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4219 00000100:00080000:3.0:1787829852.361942:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829852.361950:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829852.361987:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671477005312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829852.362008:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829852.362011:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829852.362012:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829852.362015:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4219 00000100:00080000:3.0:1787829852.362018:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829852.362020:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829852.362028:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829852.362037:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671477005440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829852.362048:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829852.362049:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477005312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829852.362056:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829852.362060:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829852.362066:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4219 00000100:00080000:3.0:1787829852.362071:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829852.362075:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829852.362081:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829852.362097:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671477005568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829852.362105:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829852.362107:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829852.362109:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829852.362111:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4219 00000100:00080000:3.0:1787829852.362115:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829852.362117:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829852.362122:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829852.362131:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671477005696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829852.362138:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829852.362141:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829852.362142:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829852.362146:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4219 00000100:00080000:3.0:1787829852.362149:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829852.362152:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829852.362156:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829852.362162:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829852.362174:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671477005824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829852.362190:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829852.362206:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829852.362225:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829852.362253:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829853.385986:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829854.409997:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829855.434018:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829855.434101:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787829855.434140:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829855.434156:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829855.434196:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829855.434205:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829855.434214:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829855.434226:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:2.0:1787829855.434245:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829855] req@ffff9a78a67743c0 x1874671477005440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829855.434273:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671477005440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829855.434285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829855.434291:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829855.434295:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829855.434303:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829855] req@ffff9a78a6774f00 x1874671477005568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829855.434345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671477005568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829855.434354:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829855.434357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829855.434362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829855.434370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829855] req@ffff9a78a6777840 x1874671477005696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829855.434384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671477005696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829855.434396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829855.434400:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829855.434404:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829855.434411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829855] req@ffff9a78a6777c00 x1874671477005312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829855.434422:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671477005312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829855.434430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829855.434434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829855.434437:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477005312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829855.434471:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829852/real 1787829855] req@ffff9a78a6775e00 x1874671477005824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829855.434485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671477005824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829855.434495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829855.434499:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829855.434502:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477005824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829856.458041:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829857.481990:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829857.481998:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829857.482002:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829857.482010:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4224 00000100:00080000:1.0:1787829857.482019:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829857.482034:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829857.482047:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829857.482080:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477005952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829857.482109:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829857.482113:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829857.482115:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829857.482119:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4224 00000100:00080000:1.0:1787829857.482123:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829857.482127:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829857.482137:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829857.482151:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477006080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829857.482162:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829857.482166:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477005952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829857.482167:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829857.482171:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829857.482180:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4224 00000100:00080000:1.0:1787829857.482185:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829857.482189:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829857.482194:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829857.482210:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477006208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829857.482221:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829857.482224:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829857.482226:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829857.482229:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4224 00000100:00080000:1.0:1787829857.482234:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787829857.482239:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829857.482244:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829857.482251:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829857.482266:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477006336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829857.482276:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829857.482280:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829857.482282:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829857.482286:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4224 00000100:00080000:1.0:1787829857.482291:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829857.482297:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829857/real 1787829857] req@ffff9a7890b1f480 x1874671477005952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829857.482298:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829857.482306:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829857.482343:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477006464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829857.482344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477005952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829857.482374:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829857.482382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829857.482388:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477005952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829857.482416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829857.482453:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829857.482468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829857/real 1787829857] req@ffff9a7890b1fc00 x1874671477006080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829857.482482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477006080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829857.482491:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829857.482495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829857.482498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829857.482505:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829857.482524:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829857.482535:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829857/real 1787829857] req@ffff9a7890b1f0c0 x1874671477006208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829857.482547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477006208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829857.482562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829857.482566:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829857.482570:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829857.482576:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829857.482596:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829857.482607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829857/real 1787829857] req@ffff9a7890b1f840 x1874671477006336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829857.482619:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477006336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829857.482629:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829857.482633:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829857.482636:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829857.482643:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829857.482661:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829857.482671:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829857/real 1787829857] req@ffff9a7890b1c000 x1874671477006464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829857.482683:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477006464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829857.482693:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829857.482696:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829857.482700:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829858.506088:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829859.529995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829860.554101:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829861.578056:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829862.601998:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829862.602009:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829862.602014:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787829862.602021:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829862.602025:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4230 00000100:00080000:3.0:1787829862.602037:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829862.602044:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829862.602082:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477006592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829862.602108:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829862.602113:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829862.602115:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829862.602119:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4230 00000100:00080000:3.0:1787829862.602123:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829862.602128:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829862.602138:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829862.602150:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477006592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829862.602154:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477006720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829862.602167:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829862.602170:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829862.602172:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829862.602177:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4230 00000100:00080000:3.0:1787829862.602181:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829862.602186:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829862.602192:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829862.602207:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829862.602207:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477006848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829862.602221:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829862.602226:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787829862.602228:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829862.602232:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4230 00000100:00080000:3.0:1787829862.602237:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829862.602241:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787829862.602247:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829862.602257:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829862/real 1787829862] req@ffff9a78a6776d00 x1874671477006592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829862.602271:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671477006976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829862.602275:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477006592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829862.602310:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829862.602311:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829862.602316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787829862.602318:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:2.0:1787829862.602322:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477006592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787829862.602324:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829862.602334:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4230 00000100:00080000:3.0:1787829862.602339:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829862.602343:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829862.602349:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006720:192.168.203.140@tcp:38:kworker.0 00000008:00100000:3.0:1787829862.602350:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829862.602369:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671477007104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829862.602375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829862.602388:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829862/real 1787829862] req@ffff9a78a67761c0 x1874671477006720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829862.602399:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477006720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829862.602405:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829862.602408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829862.602410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829862.602414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787829862.602424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829862.602430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829862/real 1787829862] req@ffff9a78a6774780 x1874671477006848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829862.602439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477006848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829862.602447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829862.602450:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829862.602454:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787829862.602461:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829862.602481:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829862.602493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829862/real 1787829862] req@ffff9a78a6775680 x1874671477006976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829862.602505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671477006976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829862.602515:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829862.602519:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829862.602523:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477006976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787829862.602536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787829862.602551:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829862.602559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829862/real 1787829862] req@ffff9a78a67770c0 x1874671477007104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829862.602566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671477007104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829862.602573:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829862.602576:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787829862.602578:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829863.626034:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829864.650091:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829865.673978:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829866.697988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829867.721926:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829867.721934:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829867.721938:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829867.721947:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4235 00000100:00080000:2.0:1787829867.721956:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829867.721962:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829867.721969:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829867.721995:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477007232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829867.722015:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829867.722018:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829867.722019:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829867.722023:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4235 00000100:00080000:2.0:1787829867.722025:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829867.722028:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829867.722036:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829867.722046:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477007360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829867.722049:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477007232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829867.722060:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829867.722063:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829867.722065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829867.722068:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4235 00000100:00080000:2.0:1787829867.722070:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829867.722073:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829867.722076:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829867.722084:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477007488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829867.722091:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829867.722094:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829867.722095:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829867.722098:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4235 00000100:00080000:2.0:1787829867.722107:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829867.722109:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829867.722113:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829867.722116:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829867.722124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477007616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829867.722133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829867.722135:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829867.722137:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829867.722140:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4235 00000100:00080000:2.0:1787829867.722142:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829867.722145:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829867.722149:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829867.722165:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477007744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829867.722166:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829867/real 1787829867] req@ffff9a78841d9e00 x1874671477007232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829867.722198:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477007232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829867.722207:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829867.722210:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829867.722213:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477007232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829867.722230:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829867.722259:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829867.722269:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829867/real 1787829867] req@ffff9a78841da580 x1874671477007360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829867.722278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477007360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829867.722283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829867.722286:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829867.722288:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829867.722292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829867.722304:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829867.722310:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829867/real 1787829867] req@ffff9a78841d8000 x1874671477007488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829867.722322:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477007488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829867.722327:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829867.722329:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829867.722331:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829867.722334:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829867.722345:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829867.722351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829867/real 1787829867] req@ffff9a78841d83c0 x1874671477007616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829867.722358:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477007616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829867.722364:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829867.722365:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829867.722367:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829867.722371:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829867.722380:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829867.722385:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829867/real 1787829867] req@ffff9a78841db480 x1874671477007744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829867.722392:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477007744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829867.722403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829867.722405:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829867.722407:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477007744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829868.746080:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829869.769978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829870.794032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829871.817971:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829872.841930:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829872.841938:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829872.841941:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829872.841950:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4240 00000100:00080000:2.0:1787829872.841960:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829872.841966:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829872.841982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829872.841998:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477007872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829872.842017:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829872.842020:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829872.842021:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829872.842024:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4240 00000100:00080000:2.0:1787829872.842034:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829872.842037:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829872.842045:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829872.842049:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477007872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829872.842056:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477008000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829872.842063:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829872.842066:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829872.842068:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829872.842070:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4240 00000100:00080000:2.0:1787829872.842072:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829872.842075:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829872.842082:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829872.842090:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477008128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829872.842096:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829872.842098:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829872.842100:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829872.842102:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4240 00000100:00080000:2.0:1787829872.842104:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829872.842106:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829872.842110:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829872.842115:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829872.842118:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477008256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829872.842126:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829872.842129:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829872.842130:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829872.842133:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4240 00000100:00080000:2.0:1787829872.842141:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829872.842144:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829872.842147:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829872.842155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477008384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829872.842185:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829872/real 1787829872] req@ffff9a78841db840 x1874671477007872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829872.842205:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477007872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829872.842229:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829872.842234:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829872.842239:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477007872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829872.842262:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829872.842285:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829872.842298:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829872/real 1787829872] req@ffff9a78841d92c0 x1874671477008000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829872.842311:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477008000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829872.842319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829872.842323:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829872.842326:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829872.842333:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829872.842360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829872.842372:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829872/real 1787829872] req@ffff9a78841d9680 x1874671477008128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829872.842384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477008128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829872.842393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829872.842396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829872.842400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829872.842406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829872.842425:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829872.842437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829872/real 1787829872] req@ffff9a78841d8b40 x1874671477008256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829872.842448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477008256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829872.842458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829872.842461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829872.842465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829872.842470:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829872.842487:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829872.842498:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829872/real 1787829872] req@ffff9a78841da940 x1874671477008384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829872.842510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477008384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829872.842520:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829872.842523:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829872.842526:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829873.866006:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829874.889969:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829875.914007:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829876.938129:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829877.961928:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829877.961936:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829877.961940:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829877.961948:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4245 00000100:00080000:2.0:1787829877.961957:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829877.961964:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829877.961981:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829877.961997:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477008512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829877.962018:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829877.962020:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829877.962022:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829877.962024:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4245 00000100:00080000:2.0:1787829877.962027:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829877.962030:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829877.962037:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829877.962045:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477008640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829877.962050:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477008512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829877.962059:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829877.962062:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829877.962063:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829877.962066:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4245 00000100:00080000:2.0:1787829877.962068:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829877.962070:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829877.962073:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829877.962081:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477008768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829877.962089:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829877.962092:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829877.962093:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829877.962096:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4245 00000100:00080000:2.0:1787829877.962101:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829877.962103:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829877.962106:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829877.962110:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829877.962119:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477008896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829877.962126:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829877.962129:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829877.962130:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829877.962133:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4245 00000100:00080000:2.0:1787829877.962135:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829877.962138:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829877.962140:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829877.962148:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477009024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829877.962160:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829877/real 1787829877] req@ffff9a78841da940 x1874671477008512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829877.962188:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477008512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829877.962197:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829877.962199:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829877.962203:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477008512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829877.962221:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829877.962238:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829877.962246:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829877/real 1787829877] req@ffff9a78841d8b40 x1874671477008640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829877.962254:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477008640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829877.962264:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829877.962266:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829877.962268:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829877.962272:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829877.962282:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829877.962288:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829877/real 1787829877] req@ffff9a78841d9680 x1874671477008768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829877.962295:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477008768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829877.962300:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829877.962302:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829877.962304:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829877.962307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829877.962317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829877.962322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829877/real 1787829877] req@ffff9a78841d92c0 x1874671477008896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829877.962329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477008896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829877.962335:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829877.962337:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829877.962339:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477008896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829877.962343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829877.962352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829877.962357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829877/real 1787829877] req@ffff9a78841db840 x1874671477009024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829877.962364:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477009024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829877.962369:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829877.962371:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829877.962373:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829878.986023:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829880.010010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829881.033988:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829882.057958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829883.081944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829883.081952:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829883.081956:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829883.081964:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4250 00010000:00010000:0.0:1787829883.081968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829883.081974:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829883.081982:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829883.082020:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477009152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829883.082041:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829883.082044:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829883.082045:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829883.082048:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4250 00000100:00080000:2.0:1787829883.082050:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829883.082053:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829883.082061:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829883.082071:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477009280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829883.082081:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477009152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829883.082084:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829883.082095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829883.082097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829883.082101:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4250 00000100:00080000:2.0:1787829883.082103:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829883.082105:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829883.082109:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829883.082121:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477009408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829883.082129:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829883.082133:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829883.082134:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829883.082152:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4250 00000100:00080000:2.0:1787829883.082175:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829883.082179:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829883.082183:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829883.082189:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829883.082196:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477009536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829883.082208:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829883.082211:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829883.082212:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829883.082217:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4250 00000100:00080000:2.0:1787829883.082220:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829883.082223:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829883.082228:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829883.082236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477009664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829883.082266:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff9a78841db480 x1874671477009152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829883.082286:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477009152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829883.082296:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829883.082300:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829883.082303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477009152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829883.082327:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829883.082346:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829883.082356:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff9a78841d83c0 x1874671477009280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829883.082365:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477009280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829883.082371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829883.082373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829883.082375:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829883.082382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829883.082401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829883.082413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff9a78841d8000 x1874671477009408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829883.082428:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477009408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829883.082437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829883.082441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829883.082443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829883.082449:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829883.082466:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829883.082474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff9a78841da580 x1874671477009536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829883.082482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477009536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829883.082490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829883.082493:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829883.082495:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829883.082499:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829883.082510:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829883.082515:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829883/real 1787829883] req@ffff9a78841d9e00 x1874671477009664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829883.082522:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477009664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829883.082529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829883.082531:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829883.082533:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829884.105953:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829885.129963:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829886.153916:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829887.177963:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829888.201935:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829888.201944:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829888.201948:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829888.201956:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4255 00000100:00080000:0.0:1787829888.201966:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829888.201971:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787829888.201994:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829888.202002:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477009792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829888.202022:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829888.202024:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829888.202026:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829888.202028:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4255 00000100:00080000:0.0:1787829888.202031:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829888.202033:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829888.202040:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829888.202051:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477009920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829888.202064:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477009792:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829888.202071:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829888.202075:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829888.202076:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829888.202080:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4255 00000100:00080000:0.0:1787829888.202082:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829888.202085:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829888.202088:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829888.202099:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477010048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829888.202120:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829888.202123:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829888.202125:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829888.202128:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4255 00000100:00080000:0.0:1787829888.202131:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829888.202134:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829888.202139:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829888.202144:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829888.202150:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477010176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829888.202159:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829888.202162:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829888.202163:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829888.202166:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4255 00000100:00080000:0.0:1787829888.202169:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829888.202171:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829888.202174:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829888.202182:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477010304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829888.202203:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff9a7890a3c3c0 x1874671477009792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829888.202224:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477009792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829888.202236:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829888.202241:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829888.202245:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477009792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829888.202269:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829888.202294:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829888.202309:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff9a7890a3fc00 x1874671477009920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829888.202322:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477009920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829888.202331:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829888.202334:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829888.202338:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477009920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829888.202345:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829888.202365:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829888.202376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff9a7890a3c000 x1874671477010048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829888.202387:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477010048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829888.202395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829888.202398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829888.202402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829888.202408:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829888.202428:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829888.202453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff9a7890a3de00 x1874671477010176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829888.202467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477010176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829888.202477:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829888.202481:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829888.202484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829888.202490:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829888.202509:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829888.202519:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829888/real 1787829888] req@ffff9a7890a3f480 x1874671477010304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829888.202531:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477010304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829888.202541:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829888.202544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829888.202548:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829889.225989:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829890.249984:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829891.273989:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829892.298007:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829893.321932:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829893.321940:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829893.321944:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829893.321952:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4260 00000100:00080000:2.0:1787829893.321962:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829893.321968:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787829893.321991:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829893.321999:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477010432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829893.322020:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829893.322023:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829893.322024:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829893.322027:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4260 00000100:00080000:2.0:1787829893.322029:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829893.322032:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829893.322039:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829893.322048:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477010560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829893.322054:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477010432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829893.322059:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829893.322062:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829893.322063:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829893.322084:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4260 00000100:00080000:2.0:1787829893.322086:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829893.322089:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829893.322093:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829893.322105:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477010688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829893.322112:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829893.322114:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829893.322116:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829893.322119:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4260 00000100:00080000:2.0:1787829893.322122:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829893.322124:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829893.322130:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829893.322131:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829893.322143:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477010816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829893.322149:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829893.322152:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829893.322153:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829893.322155:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4260 00000100:00080000:2.0:1787829893.322158:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829893.322161:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829893.322163:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829893.322172:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477010944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829893.322190:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9a78841d8780 x1874671477010432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829893.322211:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477010432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829893.322226:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829893.322229:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829893.322232:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477010432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829893.322251:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829893.322269:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829893.322279:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9a78841d9a40 x1874671477010560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829893.322287:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477010560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829893.322293:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829893.322296:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829893.322298:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829893.322302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829893.322313:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829893.322319:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9a78841da1c0 x1874671477010688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829893.322327:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477010688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829893.322334:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829893.322337:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829893.322341:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829893.322347:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829893.322367:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829893.322379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9a78841db0c0 x1874671477010816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829893.322392:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477010816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829893.322399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829893.322401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829893.322403:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829893.322407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829893.322431:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829893.322439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829893/real 1787829893] req@ffff9a78841dbc00 x1874671477010944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829893.322446:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477010944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829893.322453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829893.322455:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829893.322457:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477010944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787829894.346057:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829895.370008:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829896.394095:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829897.418077:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829898.441988:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829898.442000:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829898.442005:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829898.442017:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4265 00000100:00080000:2.0:1787829898.442044:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829898.442055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829898.442057:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829898.442102:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477011072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829898.442125:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829898.442128:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829898.442130:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829898.442133:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4265 00000100:00080000:2.0:1787829898.442136:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829898.442140:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829898.442148:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829898.442157:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477011200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829898.442169:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829898.442170:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477011072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829898.442178:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829898.442183:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829898.442189:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4265 00000100:00080000:2.0:1787829898.442193:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829898.442199:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829898.442206:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829898.442222:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477011328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829898.442242:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829898.442246:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829898.442249:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829898.442257:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829898.442263:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4265 00000100:00080000:2.0:1787829898.442279:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829898.442286:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829898.442295:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829898.442313:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477011456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829898.442323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9a78841dbc00 x1874671477011072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829898.442334:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829898.442345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477011072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829898.442353:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829898.442360:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829898.442366:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829898.442375:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4265 00000100:00080000:3.0:1787829898.442377:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829898.442381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477011072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829898.442389:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829898.442396:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829898.442401:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829898.442415:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477011584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829898.442418:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829898.442450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829898.442465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9a78841db0c0 x1874671477011200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829898.442474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477011200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829898.442480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829898.442483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829898.442486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829898.442489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829898.442500:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829898.442505:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9a78841da1c0 x1874671477011328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829898.442513:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477011328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829898.442523:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829898.442525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829898.442527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829898.442533:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829898.442553:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829898.442565:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9a78841d9a40 x1874671477011456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829898.442578:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477011456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829898.442588:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829898.442592:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829898.442594:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829898.442608:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829898.442623:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829898.442631:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829898/real 1787829898] req@ffff9a78841d8780 x1874671477011584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829898.442638:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477011584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829898.442645:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829898.442648:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829898.442650:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829899.466006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829900.490072:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829901.513997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829902.537982:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829903.561923:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829903.561936:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829903.561941:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829903.561960:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4271 00000100:00080000:2.0:1787829903.561976:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829903.561984:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829903.562022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829903.562037:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477011712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829903.562069:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829903.562072:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829903.562074:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829903.562079:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4271 00000100:00080000:2.0:1787829903.562082:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829903.562087:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829903.562098:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829903.562109:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477011712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829903.562112:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477011840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829903.562126:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829903.562129:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829903.562131:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829903.562135:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4271 00000100:00080000:2.0:1787829903.562139:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829903.562142:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829903.562147:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829903.562159:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477011968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829903.562169:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829903.562173:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829903.562175:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829903.562178:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4271 00000100:00080000:2.0:1787829903.562182:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829903.562187:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829903.562188:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829903.562195:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829903.562206:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477012096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829903.562216:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829903.562219:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829903.562221:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829903.562225:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4271 00000100:00080000:2.0:1787829903.562229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829903.562233:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829903.562238:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829903.562248:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829903/real 1787829903] req@ffff9a78841d9e00 x1874671477011712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829903.562250:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477012224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829903.562291:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477011712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829903.562303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829903.562308:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829903.562312:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477011712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829903.562340:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829903.562362:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829903.562374:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829903/real 1787829903] req@ffff9a78841da580 x1874671477011840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829903.562386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477011840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829903.562393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829903.562397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829903.562400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829903.562406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829903.562424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829903.562435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829903/real 1787829903] req@ffff9a78841d8000 x1874671477011968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829903.562447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477011968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829903.562455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829903.562457:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829903.562461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477011968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829903.562466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829903.562484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829903.562495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829903/real 1787829903] req@ffff9a78841d83c0 x1874671477012096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829903.562507:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477012096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829903.562515:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829903.562518:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829903.562521:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829903.562527:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829903.562552:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829903.562563:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829903/real 1787829903] req@ffff9a78841db480 x1874671477012224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829903.562573:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477012224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829903.562583:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829903.562586:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829903.562589:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829904.585999:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829905.610058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829906.634066:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829907.658054:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829908.681926:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829908.681933:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829908.681937:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829908.681947:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4276 00000100:00080000:1.0:1787829908.681970:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829908.681977:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829908.682012:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477012352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829908.682019:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829908.682039:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829908.682043:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829908.682045:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829908.682048:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4276 00000100:00080000:1.0:1787829908.682051:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829908.682055:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829908.682064:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829908.682077:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477012480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829908.682086:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477012352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829908.682090:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829908.682095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829908.682097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829908.682104:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4276 00000100:00080000:1.0:1787829908.682108:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829908.682112:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829908.682116:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829908.682128:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477012608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829908.682137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829908.682141:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829908.682145:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829908.682146:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829908.682153:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4276 00000100:00080000:1.0:1787829908.682157:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829908.682161:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829908.682167:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829908.682180:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477012736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829908.682188:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829908.682192:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829908.682193:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829908.682198:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4276 00000100:00080000:1.0:1787829908.682201:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829908.682207:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829908.682208:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829908/real 1787829908] req@ffff9a7890b1c000 x1874671477012352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787829908.682215:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829908.682234:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477012352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829908.682242:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477012864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829908.682246:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829908.682255:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829908.682262:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477012352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829908.682288:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829908.682309:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829908.682322:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829908/real 1787829908] req@ffff9a7890b1f840 x1874671477012480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829908.682334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477012480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829908.682342:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829908.682345:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829908.682348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829908.682354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829908.682371:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829908.682381:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829908/real 1787829908] req@ffff9a7890b1f0c0 x1874671477012608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829908.682391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477012608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829908.682398:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829908.682401:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829908.682403:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829908.682419:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829908.682438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829908.682447:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829908/real 1787829908] req@ffff9a7890b1fc00 x1874671477012736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829908.682457:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477012736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829908.682464:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829908.682477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829908.682479:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829908.682485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829908.682504:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829908.682515:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829908/real 1787829908] req@ffff9a7890b1f480 x1874671477012864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829908.682524:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477012864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829908.682539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829908.682542:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829908.682545:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477012864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829909.706034:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829910.730061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829911.754015:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829912.778011:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829913.801963:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829913.801974:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829913.801978:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829913.801988:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4281 00000100:00080000:1.0:1787829913.801999:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829913.802006:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829913.802042:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477012992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829913.802045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829913.802073:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829913.802077:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829913.802078:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829913.802081:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4281 00000100:00080000:1.0:1787829913.802085:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829913.802088:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829913.802097:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829913.802108:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477013120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829913.802117:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829913.802121:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829913.802122:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829913.802126:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4281 00000100:00080000:1.0:1787829913.802130:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829913.802133:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829913.802137:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829913.802145:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477013248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829913.802154:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829913.802158:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829913.802160:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829913.802166:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4281 00000100:00080000:1.0:1787829913.802169:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829913.802173:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829913.802179:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829913.802194:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477012992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829913.802195:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477013376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829913.802219:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829913.802222:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829913.802224:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829913.802228:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4281 00000100:00080000:1.0:1787829913.802234:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829913.802239:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829913.802244:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829913.802256:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477013504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829913.802297:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829913.802368:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829913/real 1787829913] req@ffff9a7890b1da40 x1874671477012992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829913.802389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477012992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829913.802402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829913.802407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829913.802411:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477012992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829913.802420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829913.802443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829913.802457:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829913/real 1787829913] req@ffff9a7890b1d680 x1874671477013120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829913.802469:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477013120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829913.802484:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829913.802487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829913.802491:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829913.802497:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829913.802517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829913.802528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829913/real 1787829913] req@ffff9a7890b1de00 x1874671477013248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829913.802540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477013248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829913.802548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829913.802551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829913.802554:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829913.802583:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829913.802603:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829913.802615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829913/real 1787829913] req@ffff9a7890b1e580 x1874671477013376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829913.802626:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477013376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829913.802636:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829913.802639:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829913.802642:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829913.802648:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829913.802668:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829913.802679:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829913/real 1787829913] req@ffff9a7890b1e940 x1874671477013504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829913.802690:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477013504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829913.802700:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829913.802703:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829913.802706:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829914.825995:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829915.850041:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829916.873962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829917.898027:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829918.921966:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829918.921977:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829918.921981:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829918.921991:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4286 00000100:00080000:2.0:1787829918.922003:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829918.922010:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829918.922022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829918.922054:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477013632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829918.922081:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829918.922088:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829918.922090:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829918.922094:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4286 00000100:00080000:2.0:1787829918.922098:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829918.922102:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829918.922111:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829918.922123:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477013632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829918.922124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477013760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829918.922138:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829918.922148:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829918.922149:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829918.922153:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4286 00000100:00080000:2.0:1787829918.922155:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829918.922158:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829918.922161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829918.922170:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477013888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829918.922177:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829918.922180:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829918.922181:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829918.922183:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4286 00000100:00080000:2.0:1787829918.922191:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829918.922193:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829918.922197:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829918.922205:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477014016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829918.922212:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829918.922214:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829918.922215:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829918.922217:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4286 00000100:00080000:2.0:1787829918.922219:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829918.922221:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829918.922224:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829918.922232:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477014144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829918.922261:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829918.922292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829918.922312:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829918.922333:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829919.946016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829920.970035:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829921.994011:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829921.994144:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787829921.994185:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829921.994201:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829921.994243:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829921.994254:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829921.994263:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787829921.994277:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829921.994296:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829918/real 1787829921] req@ffff9a78841d92c0 x1874671477013760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829921.994325:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477013760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829921.994351:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829921.994357:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829921.994362:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829921.994371:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829918/real 1787829921] req@ffff9a78841d9680 x1874671477013888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829921.994385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477013888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829921.994399:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829921.994404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829921.994407:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477013888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829921.994414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829918/real 1787829921] req@ffff9a78841d8b40 x1874671477014016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829921.994427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477014016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829921.994438:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829921.994441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829921.994444:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829921.994452:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829918/real 1787829921] req@ffff9a78841da940 x1874671477014144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829921.994464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477014144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829921.994476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829921.994480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829921.994484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014144:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829921.994491:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829918/real 1787829921] req@ffff9a78841db840 x1874671477013632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829921.994505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477013632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829921.994513:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829921.994517:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829921.994521:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477013632:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787829923.017997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829924.041965:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.041975:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829924.041979:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829924.041991:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4291 00000100:00080000:0.0:1787829924.042002:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829924.042008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829924.042009:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829924.042044:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477014272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829924.042066:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.042069:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829924.042070:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829924.042073:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4291 00000100:00080000:0.0:1787829924.042075:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829924.042078:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829924.042088:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829924.042096:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477014272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829924.042101:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477014400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829924.042110:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.042112:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829924.042113:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829924.042116:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4291 00000100:00080000:0.0:1787829924.042118:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829924.042120:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829924.042123:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829924.042135:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477014528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829924.042143:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.042147:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829924.042149:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829924.042154:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4291 00000100:00080000:0.0:1787829924.042168:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829924.042172:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829924.042178:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787829924.042179:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829924.042194:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477014656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829924.042204:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.042207:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829924.042209:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829924.042214:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4291 00000100:00080000:0.0:1787829924.042218:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829924.042222:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829924.042227:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829924.042239:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477014784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829924.042254:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9a7890a3da40 x1874671477014272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.042276:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477014272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.042287:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.042292:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829924.042295:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477014272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829924.042327:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829924.042353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829924.042366:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9a7890a3f480 x1874671477014400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.042376:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477014400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.042383:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.042385:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829924.042389:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829924.042394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829924.042411:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829924.042421:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9a7890a3de00 x1874671477014528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.042432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477014528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.042440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.042443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829924.042446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829924.042451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829924.042470:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829924.042479:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9a7890a3c000 x1874671477014656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.042496:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477014656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.042505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.042508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829924.042510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829924.042515:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829924.042531:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829924.042539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9a7890a3fc00 x1874671477014784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.042549:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477014784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.042559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.042562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829924.042565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477014784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829925.066019:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829926.089959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829927.114047:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829928.137944:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829929.161945:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787829929.161946:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829929.161955:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829929.161959:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829929.161970:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4296 00000100:00080000:1.0:1787829929.161980:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829929.161986:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829929.162020:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477014912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829929.162051:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829929.162055:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829929.162057:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829929.162061:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4296 00000100:00080000:1.0:1787829929.162065:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829929.162069:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829929.162078:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829929.162089:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477014912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829929.162094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477015040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829929.162105:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829929.162109:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829929.162110:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829929.162114:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4296 00000100:00080000:1.0:1787829929.162130:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829929.162134:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829929.162138:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787829929.162142:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829929.162157:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477015168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829929.162166:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829929.162169:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829929.162171:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829929.162175:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4296 00000100:00080000:1.0:1787829929.162179:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829929.162182:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9a7890b1e940 x1874671477014912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829929.162186:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829929.162196:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829929.162202:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477014912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829929.162209:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477015296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829929.162230:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829929.162231:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829929.162238:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787829929.162239:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787829929.162240:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829929.162246:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477014912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829929.162247:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4296 00000100:00080000:1.0:1787829929.162255:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829929.162259:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829929.162263:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829929.162274:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829929.162275:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477015424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829929.162302:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829929.162315:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9a7890b1e580 x1874671477015040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829929.162323:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477015040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829929.162328:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829929.162330:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829929.162332:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829929.162341:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829929.162354:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829929.162360:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9a7890b1de00 x1874671477015168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829929.162366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477015168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829929.162371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829929.162373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829929.162375:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829929.162378:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829929.162391:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829929.162397:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9a7890b1d680 x1874671477015296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829929.162403:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477015296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829929.162408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829929.162410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829929.162412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829929.162421:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829929.162430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829929.162435:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829929/real 1787829929] req@ffff9a7890b1da40 x1874671477015424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829929.162442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477015424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829929.162447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829929.162448:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829929.162450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829930.186015:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829931.210037:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829932.233950:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829933.258062:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829934.281969:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829934.281978:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829934.281983:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829934.281992:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4301 00000100:00080000:2.0:1787829934.282009:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829934.282015:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829934.282021:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829934.282051:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477015552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829934.282075:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829934.282078:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829934.282080:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829934.282084:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4301 00000100:00080000:2.0:1787829934.282088:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829934.282093:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829934.282101:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829934.282110:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477015552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829934.282117:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477015680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829934.282128:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829934.282131:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829934.282133:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829934.282136:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4301 00000100:00080000:2.0:1787829934.282140:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829934.282144:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829934.282148:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829934.282159:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477015808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829934.282168:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787829934.282169:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829934.282174:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829934.282177:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829934.282182:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4301 00000100:00080000:2.0:1787829934.282186:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829934.282191:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829934.282195:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829934.282205:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477015936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829934.282219:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829934.282221:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829934.282223:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829934.282227:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4301 00000100:00080000:2.0:1787829934.282230:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829934.282232:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff9a78841db840 x1874671477015552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829934.282236:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829934.282244:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829934.282257:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477015552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829934.282260:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477016064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829934.282282:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829934.282288:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829934.282293:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477015552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829934.282315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829934.282337:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829934.282350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff9a78841da940 x1874671477015680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829934.282361:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477015680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829934.282370:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829934.282373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829934.282376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829934.282382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829934.282400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829934.282411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff9a78841d8b40 x1874671477015808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829934.282423:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477015808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829934.282435:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829934.282438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829934.282442:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829934.282448:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829934.282465:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829934.282475:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff9a78841d9680 x1874671477015936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829934.282486:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477015936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829934.282494:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829934.282497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829934.282500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477015936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829934.282505:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829934.282521:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829934.282530:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829934/real 1787829934] req@ffff9a78841d92c0 x1874671477016064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829934.282540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477016064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829934.282549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829934.282551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829934.282555:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829935.306004:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829936.330016:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829937.353961:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829938.377968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829939.401968:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.401978:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829939.401983:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.401993:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4306 00000100:00080000:0.0:1787829939.402004:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.402010:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787829939.402038:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829939.402065:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477016192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.402091:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.402095:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829939.402097:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.402102:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4306 00000100:00080000:0.0:1787829939.402106:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.402110:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829939.402121:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829939.402131:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477016192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829939.402139:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477016320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.402150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.402153:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829939.402155:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.402166:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4306 00000100:00080000:0.0:1787829939.402170:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.402174:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829939.402182:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829939.402199:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477016448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.402209:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787829939.402214:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829939.402216:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829939.402220:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.402226:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4306 00000100:00080000:0.0:1787829939.402230:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.402234:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829939.402240:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829939.402254:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477016576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.402264:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829939.402266:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9a7890a3c3c0 x1874671477016192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829939.402272:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829939.402277:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.402284:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4306 00000100:00100000:3.0:1787829939.402289:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477016192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829939.402298:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.402306:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829939.402306:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787829939.402315:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787829939.402316:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829939.402323:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477016192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787829939.402329:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477016704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829939.402354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829939.402378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829939.402391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9a7890a3fc00 x1874671477016320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829939.402402:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477016320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829939.402411:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829939.402415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829939.402418:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829939.402424:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829939.402443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829939.402454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9a7890a3c000 x1874671477016448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829939.402465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477016448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829939.402473:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829939.402476:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829939.402479:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829939.402484:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829939.402503:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829939.402511:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9a7890a3de00 x1874671477016576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829939.402519:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477016576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829939.402525:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829939.402527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829939.402529:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829939.402533:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829939.402543:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829939.402547:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9a7890a3f480 x1874671477016704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829939.402554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477016704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829939.402560:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829939.402562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829939.402564:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829940.426036:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829941.450018:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829942.474042:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829943.498543:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829944.522051:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829944.522061:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829944.522066:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.522076:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4312 00010000:00010000:1.0:1787829944.522084:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829944.522086:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.522099:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829944.522131:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477016832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829944.522150:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829944.522153:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829944.522154:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.522157:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4312 00000100:00080000:0.0:1787829944.522160:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.522163:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829944.522170:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829944.522180:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477016960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829944.522187:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829944.522196:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477016832:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787829944.522197:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829944.522201:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.522211:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4312 00000100:00080000:0.0:1787829944.522215:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.522218:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829944.522223:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829944.522238:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477017088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829944.522247:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829944.522251:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829944.522252:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.522257:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4312 00000100:00080000:0.0:1787829944.522274:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.522278:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829944.522282:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829944.522288:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787829944.522296:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477017216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829944.522303:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829944.522305:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829944.522306:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.522310:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4312 00000100:00080000:0.0:1787829944.522320:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.522323:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829944.522327:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829944.522335:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477017344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829944.522351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9a7890a3da40 x1874671477016832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.522369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477016832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.522378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.522382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829944.522384:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477016832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829944.522408:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829944.522426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829944.522438:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9a7890a3f480 x1874671477016960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.522459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477016960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.522468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.522472:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829944.522476:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477016960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829944.522482:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829944.522504:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829944.522513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9a7890a3de00 x1874671477017088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.522520:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477017088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.522526:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.522528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829944.522530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829944.522534:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829944.522545:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829944.522550:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9a7890a3c000 x1874671477017216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.522557:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477017216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.522563:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.522565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829944.522566:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829944.522570:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829944.522579:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829944.522584:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9a7890a3fc00 x1874671477017344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.522592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477017344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.522598:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.522600:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829944.522601:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829945.546049:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829946.570043:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829947.593986:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829948.617950:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829949.641962:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829949.641973:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829949.641978:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829949.641989:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4317 00000100:00080000:2.0:1787829949.642001:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829949.642008:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787829949.642012:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829949.642050:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477017472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829949.642077:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829949.642081:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829949.642084:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829949.642089:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4317 00000100:00080000:2.0:1787829949.642094:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829949.642099:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829949.642112:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829949.642122:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477017472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829949.642130:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477017600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829949.642145:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829949.642150:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829949.642152:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829949.642157:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4317 00000100:00080000:2.0:1787829949.642177:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829949.642182:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829949.642189:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829949.642200:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829949.642208:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477017728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829949.642220:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829949.642224:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829949.642227:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829949.642231:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4317 00000100:00080000:2.0:1787829949.642235:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829949.642241:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829949.642249:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829949.642257:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9a78841db480 x1874671477017472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829949.642265:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477017856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829949.642280:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829949.642283:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477017472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829949.642285:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829949.642291:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829949.642302:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4317 00000100:00080000:3.0:1787829949.642303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829949.642310:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829949.642313:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787829949.642320:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829949.642321:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477017472:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829949.642330:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829949.642348:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477017984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829949.642353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829949.642390:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829949.642406:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9a78841d83c0 x1874671477017600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829949.642421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477017600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829949.642437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829949.642441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829949.642445:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829949.642451:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829949.642472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829949.642484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9a78841d8000 x1874671477017728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829949.642498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477017728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829949.642507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829949.642510:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829949.642514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829949.642520:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829949.642541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829949.642553:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9a78841da580 x1874671477017856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829949.642566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477017856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829949.642577:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829949.642580:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829949.642584:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829949.642600:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829949.642620:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829949.642632:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9a78841d9e00 x1874671477017984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829949.642645:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477017984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829949.642661:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829949.642665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829949.642669:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477017984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829950.665932:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829951.689950:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829952.713977:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829953.738032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829954.761954:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829954.761962:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829954.761966:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829954.761980:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4322 00000100:00080000:1.0:1787829954.761988:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829954.761994:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829954.762005:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787829954.762032:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477018112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829954.762056:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829954.762060:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829954.762061:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829954.762065:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4322 00000100:00080000:1.0:1787829954.762069:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829954.762074:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829954.762082:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829954.762095:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477018112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829954.762097:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477018240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829954.762111:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829954.762114:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829954.762116:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829954.762120:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4322 00000100:00080000:1.0:1787829954.762124:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829954.762128:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829954.762132:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829954.762144:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477018368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829954.762153:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829954.762158:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000800:00000100:3.0:1787829954.762159:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829954.762161:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829954.762168:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4322 00000100:00080000:1.0:1787829954.762172:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829954.762176:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829954.762181:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829954.762193:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477018496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829954.762201:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829954.762204:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829954.762206:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829954.762210:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4322 00000100:00080000:1.0:1787829954.762213:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829954.762217:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829954.762222:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829954.762231:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829954/real 1787829954] req@ffff9a7890b1f480 x1874671477018112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829954.762235:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477018624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829954.762274:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477018112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829954.762286:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829954.762290:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829954.762295:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477018112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829954.762316:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829954.762338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829954.762351:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829954/real 1787829954] req@ffff9a7890b1fc00 x1874671477018240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829954.762363:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477018240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829954.762371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829954.762374:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829954.762377:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829954.762383:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829954.762400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829954.762423:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829954/real 1787829954] req@ffff9a7890b1f0c0 x1874671477018368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829954.762435:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477018368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829954.762443:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829954.762446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829954.762450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829954.762456:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829954.762474:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829954.762484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829954/real 1787829954] req@ffff9a7890b1f840 x1874671477018496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829954.762494:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477018496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829954.762505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829954.762508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829954.762512:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829954.762517:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829954.762534:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829954.762543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829954/real 1787829954] req@ffff9a7890b1c000 x1874671477018624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829954.762554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477018624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829954.762563:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829954.762565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829954.762568:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829955.786056:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829956.810010:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829957.834016:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829958.857992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829959.881949:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829959.882021:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829959.882027:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829959.882031:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829959.882041:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4327 00000100:00080000:2.0:1787829959.882060:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829959.882067:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829959.882102:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477018752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829959.882129:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829959.882133:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829959.882135:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829959.882139:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4327 00000100:00080000:2.0:1787829959.882143:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829959.882147:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829959.882157:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829959.882170:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477018880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829959.882180:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477018752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829959.882197:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829959.882201:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829959.882202:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829959.882207:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4327 00000100:00080000:2.0:1787829959.882224:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829959.882229:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829959.882234:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829959.882247:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477019008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829959.882257:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787829959.882263:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829959.882265:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829959.882270:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829959.882276:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4327 00000100:00080000:2.0:1787829959.882280:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829959.882285:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829959.882290:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829959.882303:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477019136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829959.882312:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829959.882316:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829959.882318:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829959.882322:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4327 00000100:00080000:2.0:1787829959.882328:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829959.882333:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829959.882337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829959/real 1787829959] req@ffff9a78841d8780 x1874671477018752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787829959.882340:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829959.882359:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477018752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829959.882369:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477019264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829959.882370:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829959.882379:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829959.882386:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477018752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829959.882413:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829959.882433:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829959.882443:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829959/real 1787829959] req@ffff9a78841d9a40 x1874671477018880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829959.882455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477018880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829959.882463:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829959.882466:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829959.882469:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477018880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829959.882474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829959.882487:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829959.882495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829959/real 1787829959] req@ffff9a78841da1c0 x1874671477019008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829959.882505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477019008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829959.882513:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829959.882516:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829959.882518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829959.882523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829959.882535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829959.882543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829959/real 1787829959] req@ffff9a78841db0c0 x1874671477019136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829959.882554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477019136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829959.882562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829959.882565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829959.882567:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829959.882572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829959.882584:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829959.882592:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829959/real 1787829959] req@ffff9a78841dbc00 x1874671477019264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829959.882612:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477019264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829959.882621:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829959.882624:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829959.882627:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829960.906032:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829961.930144:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829962.954091:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829963.978099:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829965.001960:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829965.001976:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829965.001980:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829965.001990:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4332 00000100:00080000:1.0:1787829965.002000:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829965.002006:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787829965.002039:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477019392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829965.002066:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829965.002070:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:0.0:1787829965.002072:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829965.002073:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829965.002079:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4332 00000100:00080000:1.0:1787829965.002083:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829965.002087:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829965.002096:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829965.002107:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477019520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829965.002116:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829965.002120:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829965.002122:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829965.002129:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4332 00000100:00100000:3.0:1787829965.002132:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477019392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787829965.002134:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829965.002141:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829965.002145:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829965.002156:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477019648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829965.002165:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829965.002169:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787829965.002171:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829965.002175:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4332 00000100:00080000:1.0:1787829965.002188:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829965.002198:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787829965.002202:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829965.002208:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787829965.002221:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477019776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829965.002230:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829965.002233:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787829965.002235:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829965.002240:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4332 00000100:00080000:1.0:1787829965.002244:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829965.002249:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9a7890b1c000 x1874671477019392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829965.002250:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787829965.002267:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829965.002271:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477019392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829965.002282:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477019904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829965.002283:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.002292:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829965.002300:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477019392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829965.002309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829965.002332:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829965.002345:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9a7890b1f840 x1874671477019520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.002358:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477019520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.002366:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.002369:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829965.002372:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829965.002395:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829965.002414:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829965.002425:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9a7890b1f0c0 x1874671477019648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.002436:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477019648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.002443:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.002446:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829965.002450:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829965.002456:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829965.002475:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829965.002485:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9a7890b1fc00 x1874671477019776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.002496:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477019776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.002504:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.002507:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829965.002510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829965.002516:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829965.002532:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829965.002542:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9a7890b1f480 x1874671477019904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.002553:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477019904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.002562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.002565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829965.002585:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477019904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829966.026394:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829967.049955:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829968.074093:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829969.098625:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829970.122098:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829970.122108:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829970.122112:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.122121:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4337 00000100:00080000:2.0:1787829970.122130:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829970.122136:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829970.122171:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477020032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829970.122213:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787829970.122215:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829970.122218:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829970.122222:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.122227:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4337 00000100:00080000:2.0:1787829970.122232:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829970.122239:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829970.122240:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477020032:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787829970.122251:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829970.122266:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477020160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829970.122278:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829970.122284:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829970.122286:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.122293:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4337 00000800:00000100:3.0:1787829970.122294:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829970.122301:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829970.122305:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829970.122310:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829970.122322:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477020288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829970.122332:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829970.122333:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9a78841dbc00 x1874671477020032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829970.122342:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829970.122347:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.122354:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4337 00000100:00100000:3.0:1787829970.122355:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477020032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829970.122358:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829970.122366:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829970.122368:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:2.0:1787829970.122372:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787829970.122373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829970.122379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477020032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829970.122389:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477020416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829970.122398:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020160:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787829970.122401:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829970.122406:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829970.122408:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829970.122413:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4337 00000100:00080000:2.0:1787829970.122418:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829970.122418:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787829970.122426:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787829970.122431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9a78841db0c0 x1874671477020160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787829970.122432:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829970.122444:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477020160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829970.122450:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477020544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829970.122452:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829970.122458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829970.122464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829970.122473:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829970.122486:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829970.122492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9a78841da1c0 x1874671477020288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829970.122498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477020288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829970.122506:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829970.122508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829970.122510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829970.122517:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829970.122527:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829970.122555:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9a78841d9a40 x1874671477020416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829970.122563:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477020416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829970.122568:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829970.122570:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829970.122572:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829970.122576:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829970.122599:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829970.122609:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829970] req@ffff9a78841d8780 x1874671477020544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829970.122619:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477020544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829970.122628:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829970.122631:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829970.122635:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829971.146094:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829972.170347:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829973.194229:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829974.218114:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829975.242032:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.242041:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829975.242045:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.242054:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4342 00000100:00080000:0.0:1787829975.242064:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829975.242071:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829975.242102:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477020672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829975.242124:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.242127:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829975.242129:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.242133:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4342 00000100:00080000:0.0:1787829975.242136:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829975.242139:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829975.242148:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829975.242158:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477020800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829975.242167:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.242170:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829975.242172:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.242175:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4342 00000100:00080000:0.0:1787829975.242177:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829975.242180:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829975.242183:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829975.242191:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477020928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829975.242199:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.242202:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829975.242204:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.242208:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4342 00000100:00080000:0.0:1787829975.242210:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829975.242213:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829975.242218:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829975.242227:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477021056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829975.242234:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.242237:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829975.242239:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.242242:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4342 00000100:00080000:0.0:1787829975.242245:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829975.242248:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829975.242252:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787829975.242261:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477021184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829975.242928:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477020672:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787829975.243012:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829975.243090:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9a7890a3c3c0 x1874671477020672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.243111:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477020672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.243124:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.243129:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829975.243133:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477020672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829975.243141:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829975.243164:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829975.243174:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9a7890a3fc00 x1874671477020800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.243182:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477020800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.243188:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.243190:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829975.243192:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829975.243197:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829975.243207:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829975.243213:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9a7890a3c000 x1874671477020928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.243220:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477020928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.243225:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.243226:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829975.243228:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477020928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829975.243231:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829975.243242:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829975.243247:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9a7890a3de00 x1874671477021056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.243253:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477021056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.243259:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.243261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829975.243263:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829975.243266:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829975.243275:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829975.243280:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9a7890a3f480 x1874671477021184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829975.243287:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477021184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829975.243297:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829975.243300:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829975.243303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829975.243453:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829976.266591:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829977.290043:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829978.314281:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829979.338304:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829980.362030:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.362043:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829980.362049:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829980.362062:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4347 00000100:00080000:0.0:1787829980.362075:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.362083:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787829980.362128:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477021312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829980.362164:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.362168:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829980.362171:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829980.362176:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4347 00000100:00080000:0.0:1787829980.362180:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.362186:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829980.362198:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829980.362209:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477021312:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787829980.362209:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787829980.362219:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477021440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829980.362234:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.362239:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829980.362242:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829980.362247:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4347 00000100:00080000:0.0:1787829980.362251:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.362254:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829980.362261:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829980.362276:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477021568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829980.362287:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.362291:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787829980.362293:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829980.362300:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4347 00000800:00000100:1.0:1787829980.362303:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787829980.362307:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.362314:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829980.362320:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787829980.362335:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477021696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829980.362346:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829980.362349:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787829980.362352:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829980.362357:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9a7890a3da40 x1874671477021312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829980.362361:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4347 00000100:00080000:0.0:1787829980.362373:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829980.362386:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787829980.362396:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787829980.362401:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477021312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829980.362415:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477021824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829980.362417:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829980.362428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787829980.362448:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477021312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787829980.362493:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787829980.362523:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829980.362536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9a7890a3f480 x1874671477021440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.362556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477021440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829980.362563:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829980.362567:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787829980.362570:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829980.362577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787829980.362596:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829980.362608:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9a7890a3de00 x1874671477021568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.362620:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477021568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829980.362633:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829980.362636:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787829980.362640:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829980.362645:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787829980.362660:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829980.362668:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9a7890a3c000 x1874671477021696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.362679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477021696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829980.362689:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829980.362691:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787829980.362694:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787829980.362700:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787829980.362722:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787829980.362737:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9a7890a3fc00 x1874671477021824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.362748:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477021824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829980.362757:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829980.362760:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787829980.362763:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477021824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787829981.386098:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829982.410314:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829983.433954:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829984.458021:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829985.481993:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829985.482005:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829985.482010:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829985.482023:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4352 00000100:00080000:2.0:1787829985.482036:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829985.482044:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787829985.482087:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477021952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829985.482119:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829985.482123:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829985.482124:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829985.482128:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4352 00000100:00080000:2.0:1787829985.482131:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829985.482135:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829985.482146:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829985.482158:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477022080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829985.482165:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477021952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829985.482171:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829985.482175:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829985.482177:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829985.482182:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4352 00000100:00080000:2.0:1787829985.482185:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829985.482189:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829985.482193:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829985.482204:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477022208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829985.482219:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829985.482222:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829985.482224:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829985.482228:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4352 00000100:00080000:2.0:1787829985.482231:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829985.482235:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829985.482240:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787829985.482252:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477022336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829985.482261:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829985.482264:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829985.482266:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829985.482269:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4352 00000100:00080000:2.0:1787829985.482272:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829985.482275:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787829985.482278:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829985.482285:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829985.482296:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477022464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829985.482303:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787829985.482324:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787829985.482341:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829985.482709:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829986.506536:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829987.530547:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829988.554737:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829988.554744:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787829988.554803:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829988.554823:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829988.555395:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829988.555423:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829988.555434:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787829988.555444:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787829988.555525:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829988] req@ffff9a78841db480 x1874671477022464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829988.555563:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477022464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829988.555581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829988.555586:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829988.555591:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022464:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829988.555600:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829988] req@ffff9a78841da580 x1874671477022080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829988.555613:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477022080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829988.555620:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829988.555623:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829988.555627:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829988.555633:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829988] req@ffff9a78841d8000 x1874671477022208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829988.555644:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477022208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829988.555651:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829988.555654:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829988.555657:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829988.555663:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829988] req@ffff9a78841d83c0 x1874671477022336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829988.555674:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477022336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829988.555683:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829988.555687:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829988.555690:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829988.555696:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829988] req@ffff9a78841d9e00 x1874671477021952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829988.555710:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477021952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829988.555719:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829988.555723:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829988.555726:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477021952:192.168.203.140@tcp:250:kworker.0 00010000:00010000:3.0:1787829989.578043:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829990.601922:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829990.601932:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829990.601936:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829990.601951:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4358 00000100:00080000:2.0:1787829990.601961:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829990.601968:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787829990.601993:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829990.602000:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477022592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829990.602021:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829990.602025:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829990.602026:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829990.602030:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4358 00000100:00080000:2.0:1787829990.602034:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829990.602038:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829990.602048:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829990.602056:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477022592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829990.602061:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477022720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829990.602072:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829990.602075:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829990.602077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829990.602092:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4358 00000100:00080000:2.0:1787829990.602107:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829990.602111:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787829990.602116:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787829990.602118:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829990.602131:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477022848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829990.602140:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829990.602144:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829990.602146:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829990.602150:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4358 00000100:00080000:2.0:1787829990.602154:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829990.602158:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829990.602163:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787829990.602165:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9a78841d92c0 x1874671477022592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829990.602174:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477022976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829990.602186:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829990.602189:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477022592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829990.602191:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829990.602193:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829990.602201:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4358 00000100:00080000:2.0:1787829990.602207:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829990.602208:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829990.602212:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787829990.602217:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:2.0:1787829990.602218:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787829990.602224:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477022592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787829990.602231:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477023104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829990.602267:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829990.602291:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829990.602303:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9a78841d9680 x1874671477022720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829990.602315:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477022720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829990.602322:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829990.602325:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829990.602328:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829990.602334:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829990.602669:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829990.602687:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9a78841d8b40 x1874671477022848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829990.602707:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477022848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829990.602714:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829990.602717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829990.602720:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829990.602727:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829990.602748:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829990.602759:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9a78841da940 x1874671477022976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829990.602770:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477022976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829990.602779:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829990.602782:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829990.602785:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477022976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829990.602803:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829990.602822:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829990.602856:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9a78841db840 x1874671477023104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829990.602868:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477023104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829990.602882:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829990.602885:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829990.602888:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829991.626216:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829992.650611:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829993.674059:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829994.698098:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829995.721935:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829995.721942:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829995.721946:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829995.721954:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4363 00000100:00080000:2.0:1787829995.721963:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829995.721968:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787829995.721988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787829995.722014:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477023232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829995.722040:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829995.722043:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829995.722044:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829995.722046:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4363 00000100:00080000:2.0:1787829995.722049:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829995.722053:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829995.722062:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829995.722071:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477023360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829995.722079:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477023232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787829995.722087:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829995.722091:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829995.722092:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829995.722095:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4363 00000100:00080000:2.0:1787829995.722097:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829995.722099:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829995.722104:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829995.722111:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477023488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829995.722117:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829995.722120:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787829995.722121:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829995.722123:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4363 00000100:00080000:2.0:1787829995.722141:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829995.722143:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829995.722146:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787829995.722151:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787829995.722155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477023616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829995.722162:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829995.722165:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787829995.722167:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829995.722172:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4363 00000100:00080000:2.0:1787829995.722181:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829995.722183:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787829995.722186:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787829995.722198:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477023744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829995.722210:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829995/real 1787829995] req@ffff9a78841db840 x1874671477023232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829995.722224:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477023232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829995.722233:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829995.722236:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829995.722239:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477023232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787829995.722261:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829995.722276:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829995.722283:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829995/real 1787829995] req@ffff9a78841da940 x1874671477023360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829995.722290:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477023360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829995.722295:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829995.722298:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829995.722300:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829995.722304:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787829995.722313:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829995.722318:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829995/real 1787829995] req@ffff9a78841d8b40 x1874671477023488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829995.722324:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477023488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829995.722328:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829995.722330:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829995.722332:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787829995.722336:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829995.722345:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829995.722363:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829995/real 1787829995] req@ffff9a78841d9680 x1874671477023616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829995.722375:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477023616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829995.722384:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829995.722388:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829995.722391:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787829995.722397:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787829995.722416:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787829995.722424:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829995/real 1787829995] req@ffff9a78841d92c0 x1874671477023744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829995.722431:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477023744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829995.722437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829995.722439:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787829995.722441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477023744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787829996.745995:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829997.770003:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829998.794035:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829999.818494:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830000.841979:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830000.841986:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830000.841989:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830000.841998:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4368 00000100:00080000:0.0:1787830000.842005:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830000.842011:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830000.842047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477023872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830000.842063:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830000.842073:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830000.842078:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830000.842080:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830000.842085:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4368 00000100:00080000:0.0:1787830000.842089:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830000.842094:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830000.842102:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830000.842118:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477024000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830000.842121:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477023872:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830000.842134:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830000.842137:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830000.842138:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830000.842142:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4368 00000100:00080000:0.0:1787830000.842144:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830000.842147:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830000.842153:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830000.842167:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477024128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830000.842176:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830000.842180:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830000.842181:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830000.842187:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4368 00000100:00080000:0.0:1787830000.842203:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830000.842208:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830000.842213:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787830000.842216:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830000.842232:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477024256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830000.842244:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830000.842247:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830000.842248:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830000.842251:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4368 00000100:00080000:0.0:1787830000.842254:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830000.842257:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830000.842263:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830000.842276:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477024384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830000.842291:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830000/real 1787830000] req@ffff9a7890a3c3c0 x1874671477023872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830000.842313:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477023872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830000.842338:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830000.842343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830000.842347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477023872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830000.842377:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830000.842400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830000.842412:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830000/real 1787830000] req@ffff9a7890a3fc00 x1874671477024000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830000.842429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477024000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830000.842437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830000.842440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830000.842442:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830000.842455:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830000.842473:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830000.842484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830000/real 1787830000] req@ffff9a7890a3c000 x1874671477024128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830000.842494:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477024128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830000.842502:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830000.842505:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830000.842507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830000.842513:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830000.842530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830000.842539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830000/real 1787830000] req@ffff9a7890a3de00 x1874671477024256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830000.842549:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477024256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830000.842563:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830000.842566:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830000.842569:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830000.842574:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830000.842590:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830000.842600:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830000/real 1787830000] req@ffff9a7890a3f480 x1874671477024384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830000.842611:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477024384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830000.842620:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830000.842632:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830000.842635:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830001.866045:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830002.891777:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830003.915254:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830004.937957:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830005.961959:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830005.961968:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830005.961972:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830005.961982:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4373 00000100:00080000:0.0:1787830005.961990:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830005.961997:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830005.962031:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477024512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830005.962054:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830005.962057:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830005.962059:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830005.962063:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4373 00000100:00080000:0.0:1787830005.962070:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830005.962073:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830005.962082:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830005.962094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477024640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830005.962104:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830005.962108:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830005.962110:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830005.962114:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4373 00000100:00080000:0.0:1787830005.962117:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830005.962121:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830005.962124:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830005.962134:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477024768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830005.962138:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477024512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830005.962152:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830005.962155:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830005.962157:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830005.962161:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4373 00000100:00080000:0.0:1787830005.962164:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830005.962168:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830005.962173:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830005.962183:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477024896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830005.962192:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830005.962195:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830005.962197:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830005.962203:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4373 00000800:00000100:3.0:1787830005.962203:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830005.962209:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830005.962214:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830005.962218:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830005.962228:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477025024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830005.962262:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830005/real 1787830005] req@ffff9a7890a3da40 x1874671477024512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830005.962295:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477024512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830005.962306:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830005.962310:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830005.962314:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477024512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830005.962322:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830005.962347:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830005.962359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830005/real 1787830005] req@ffff9a7890a3f480 x1874671477024640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830005.962371:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477024640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830005.962379:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830005.962382:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830005.962385:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830005.962407:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830005.962426:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830005.962448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830005/real 1787830005] req@ffff9a7890a3de00 x1874671477024768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830005.962460:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477024768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830005.962468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830005.962471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830005.962474:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830005.962480:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830005.962499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830005.962510:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830005/real 1787830005] req@ffff9a7890a3c000 x1874671477024896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830005.962521:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477024896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830005.962529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830005.962532:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830005.962535:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477024896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830005.962541:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830005.962559:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830005.962570:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830005/real 1787830005] req@ffff9a7890a3fc00 x1874671477025024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830005.962580:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477025024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830005.962589:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830005.962591:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830005.962594:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830005.964517:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830006.988657:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830008.010049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830009.034045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830010.058152:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830011.082052:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830011.082091:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830011.082097:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830011.082101:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830011.082111:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4378 00000100:00080000:1.0:1787830011.082120:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830011.082125:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830011.082158:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477025152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830011.082183:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830011.082186:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830011.082188:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830011.082192:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4378 00000100:00080000:1.0:1787830011.082196:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830011.082199:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830011.082207:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830011.082219:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477025280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830011.082228:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830011.082231:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830011.082233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830011.082237:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4378 00000100:00080000:1.0:1787830011.082240:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830011.082260:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830011.082266:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830011.082277:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477025408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830011.082285:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830011.082287:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830011.082289:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830011.082292:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4378 00000100:00080000:1.0:1787830011.082295:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830011.082298:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830011.082302:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830011.082310:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477025536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830011.082316:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830011.082319:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830011.082320:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830011.082323:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4378 00000100:00080000:1.0:1787830011.082325:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830011.082328:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830011.082331:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830011.082339:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477025664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830011.082926:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477025152:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830011.082990:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830011.083042:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff9a7890b1da40 x1874671477025152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.083058:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477025152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.083066:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.083070:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830011.083074:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477025152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830011.083081:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830011.083100:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830011.083111:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff9a7890b1d680 x1874671477025280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.083121:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477025280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.083128:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.083130:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830011.083133:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830011.083138:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830011.083151:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830011.083159:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff9a7890b1de00 x1874671477025408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.083168:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477025408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.083179:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.083181:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830011.083183:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830011.083188:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830011.083201:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830011.083210:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff9a7890b1e580 x1874671477025536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.083220:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477025536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.083228:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.083231:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830011.083234:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830011.083239:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830011.084133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830011.084145:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830011/real 1787830011] req@ffff9a7890b1e940 x1874671477025664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830011.084156:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477025664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830011.084164:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830011.084167:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830011.084170:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830012.106055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830013.130082:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830014.153990:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830015.181317:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830016.201944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830016.201954:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830016.201958:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830016.201967:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4383 00000100:00080000:1.0:1787830016.201977:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830016.201983:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830016.202019:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477025792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830016.202050:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830016.202054:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830016.202056:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830016.202060:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4383 00000100:00080000:1.0:1787830016.202063:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830016.202067:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830016.202075:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787830016.202080:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830016.202090:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477025920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830016.202101:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830016.202104:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830016.202105:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830016.202109:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4383 00000100:00080000:1.0:1787830016.202112:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830016.202115:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830016.202119:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830016.202127:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477026048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830016.202136:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830016.202139:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830016.202141:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830016.202144:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4383 00000100:00080000:1.0:1787830016.202147:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830016.202150:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830016.202154:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830016.202162:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477026176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830016.202176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830016.202180:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830016.202181:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830016.202184:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4383 00000100:00080000:1.0:1787830016.202187:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830016.202189:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830016.202193:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830016.202200:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477026304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830016.203057:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477025792:192.168.203.140@tcp:250:kworker.0 00000800:00000100:2.0:1787830016.203116:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830016.203169:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9a7890b1e940 x1874671477025792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830016.203188:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477025792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830016.203200:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830016.203206:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830016.203221:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477025792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830016.203229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830016.203253:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830016.203267:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9a7890b1e580 x1874671477025920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830016.203280:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477025920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830016.203287:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830016.203291:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830016.203294:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477025920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830016.203302:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830016.203323:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830016.203334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9a7890b1de00 x1874671477026048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830016.203346:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477026048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830016.203354:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830016.203358:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830016.203361:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830016.203366:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830016.203384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830016.203394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9a7890b1d680 x1874671477026176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830016.203418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477026176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830016.203567:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830016.203571:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830016.203575:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830016.203582:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830016.203603:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830016.203613:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830016/real 1787830016] req@ffff9a7890b1da40 x1874671477026304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830016.203625:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477026304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830016.203634:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830016.203637:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830016.203639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830017.225942:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830018.250046:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830019.277129:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830020.297942:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830021.322675:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830021.323679:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.323696:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830021.323701:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.323711:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4388 00000100:00080000:2.0:1787830021.323721:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.323728:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830021.323759:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477026432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830021.323790:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.323794:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830021.323795:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.323799:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4388 00000100:00080000:2.0:1787830021.323802:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.323806:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830021.323814:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830021.323826:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477026560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830021.323868:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.323872:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830021.323874:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.323878:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4388 00000100:00080000:2.0:1787830021.323882:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.323885:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830021.323890:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830021.323901:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477026688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830021.323906:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477026432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830021.323916:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.323921:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830021.323923:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.323927:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4388 00000100:00080000:2.0:1787830021.323930:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.323934:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830021.323939:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830021.323949:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477026816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830021.323958:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830021.323961:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830021.323963:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830021.323966:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4388 00000800:00000100:3.0:1787830021.323971:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830021.323973:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830021.323979:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830021.323984:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830021.323996:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477026944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830021.324027:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9a78841d9e00 x1874671477026432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.324044:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477026432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.324068:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.324072:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830021.324076:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477026432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830021.324084:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830021.324106:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830021.324119:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9a78841d83c0 x1874671477026560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.324131:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477026560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.324139:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.324142:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830021.324145:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830021.324170:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830021.324641:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830021.324655:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9a78841d8000 x1874671477026688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.324668:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477026688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.324675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.324679:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830021.324682:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830021.324698:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830021.324718:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830021.324730:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9a78841da580 x1874671477026816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.324741:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477026816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.324750:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.324753:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830021.324756:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830021.324762:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830021.324779:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830021.324790:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830021/real 1787830021] req@ffff9a78841db480 x1874671477026944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830021.324801:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477026944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830021.324810:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830021.324813:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830021.324816:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477026944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830022.346285:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830023.369979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830024.394025:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830025.418174:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830026.441939:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830026.443199:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830026.443212:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830026.443217:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830026.443226:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4393 00000100:00080000:0.0:1787830026.443235:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830026.443241:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830026.443282:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477027072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830026.443316:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830026.443320:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830026.443322:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830026.443326:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4393 00000100:00080000:0.0:1787830026.443329:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830026.443337:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830026.443346:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830026.443358:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477027200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830026.443367:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830026.443370:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830026.443372:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830026.443376:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4393 00000100:00080000:0.0:1787830026.443378:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830026.443382:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830026.443386:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830026.443394:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477027328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830026.443402:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830026.443405:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830026.443407:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830026.443410:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4393 00000100:00080000:0.0:1787830026.443413:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830026.443416:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830026.443420:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830026.443427:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477027456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830026.443435:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830026.443437:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830026.443439:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830026.443442:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4393 00000100:00080000:0.0:1787830026.443445:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830026.443448:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830026.443451:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830026.443460:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477027584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830026.443538:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477027072:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830026.443589:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830026.443637:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9a7890a3c3c0 x1874671477027072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.443654:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477027072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.443664:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.443667:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830026.443671:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477027072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830026.443677:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830026.443696:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830026.443706:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9a7890a3fc00 x1874671477027200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.443717:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477027200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.443725:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.443728:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830026.443731:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830026.443736:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830026.443752:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830026.443760:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9a7890a3c000 x1874671477027328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.443771:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477027328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.443778:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.443781:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830026.443784:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830026.443789:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830026.443803:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830026.443812:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9a7890a3de00 x1874671477027456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.443822:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477027456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.443847:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.443850:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830026.443854:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830026.443859:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830026.443875:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830026.443886:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830026/real 1787830026] req@ffff9a7890a3f480 x1874671477027584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830026.443896:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477027584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830026.443905:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830026.443908:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830026.443911:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830027.466467:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830028.490072:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830029.513978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830030.538090:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830031.562285:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830031.562355:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830031.562365:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830031.562369:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830031.562379:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4399 00000100:00080000:1.0:1787830031.562388:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830031.562395:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830031.562431:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477027712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830031.562455:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830031.562459:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830031.562462:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830031.562465:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4399 00000100:00080000:1.0:1787830031.562469:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830031.562474:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830031.562507:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477027712:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787830031.562554:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830031.562586:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477027840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830031.562598:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830031.562599:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830031.562606:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830031.562609:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830031.562615:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4399 00000100:00080000:1.0:1787830031.562620:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830031.562627:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830031.562632:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830031.562648:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477027968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830031.562660:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830031.562666:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1787830031.562667:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830031/real 1787830031] req@ffff9a7890b1f480 x1874671477027712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830031.562669:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830031.562680:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4399 00000100:00080000:1.0:1787830031.562687:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830031.562695:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830031.562701:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830031.562704:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477027712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830031.562716:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830031.562723:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830031.562728:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477027712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830031.562729:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477028096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830031.562743:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830031.562745:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830031.562748:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787830031.562755:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027840:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830031.562756:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4399 00000100:00080000:1.0:1787830031.562767:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830031.562776:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830031.562782:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787830031.562800:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830031.562807:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477028224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830031.562813:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830031/real 1787830031] req@ffff9a7890b1fc00 x1874671477027840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830031.562925:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477027840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830031.562933:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830031.562936:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830031.562938:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830031.562943:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830031.562959:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830031.562967:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830031/real 1787830031] req@ffff9a7890b1f0c0 x1874671477027968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830031.562974:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477027968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830031.562979:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830031.562981:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830031.562983:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477027968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830031.562987:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830031.562996:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830031.563001:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830031/real 1787830031] req@ffff9a7890b1f840 x1874671477028096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830031.563012:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477028096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830031.563018:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830031.563020:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830031.563021:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830031.563853:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830031.563887:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830031.563901:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830031/real 1787830031] req@ffff9a7890b1c000 x1874671477028224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830031.563915:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477028224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830031.563926:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830031.563930:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830031.563934:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830032.586022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830033.610068:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830034.634029:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830035.658158:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830036.681964:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830036.681974:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830036.681978:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830036.681989:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4404 00000100:00080000:2.0:1787830036.682000:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830036.682008:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830036.682044:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477028352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830036.682062:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830036.682067:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830036.682080:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830036.682085:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4404 00000100:00080000:2.0:1787830036.682089:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830036.682094:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830036.682100:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477028352:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787830036.682106:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830036.682123:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477028480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830036.682140:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830036.682143:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830036.682145:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830036.682150:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4404 00000100:00080000:2.0:1787830036.682153:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830036.682159:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787830036.682160:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787830036.682166:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830036.682180:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477028608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830036.682189:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830036.682193:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830036.682195:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830036.682199:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4404 00000100:00080000:2.0:1787830036.682202:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830036.682205:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830036.682210:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830036.682222:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477028736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830036.682222:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830036/real 1787830036] req@ffff9a78841d8780 x1874671477028352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830036.682244:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830036.682248:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830036.682249:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830036.682253:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4404 00000100:00080000:2.0:1787830036.682256:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830036.682258:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477028352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830036.682260:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830036.682264:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830036.682273:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477028864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830036.682284:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830036.682291:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830036.682296:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477028352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830036.682319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830036.682341:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830036.682354:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830036/real 1787830036] req@ffff9a78841d9a40 x1874671477028480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830036.682366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477028480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830036.682374:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830036.682377:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830036.682380:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830036.682386:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830036.682405:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830036.682416:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830036/real 1787830036] req@ffff9a78841da1c0 x1874671477028608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830036.682427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477028608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830036.682435:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830036.682438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830036.682441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830036.682447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830036.682466:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830036.682477:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830036/real 1787830036] req@ffff9a78841db0c0 x1874671477028736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830036.682489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477028736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830036.682497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830036.682500:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830036.682503:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830036.682508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830036.682525:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830036.682536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830036/real 1787830036] req@ffff9a78841dbc00 x1874671477028864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830036.682548:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477028864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830036.682556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830036.682559:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830036.682563:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477028864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830036.682640:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830037.706088:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830038.730030:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830039.753996:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830040.779065:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830041.803264:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830041.803274:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830041.803295:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830041.803305:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4409 00000100:00080000:0.0:1787830041.803316:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830041.803323:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830041.803360:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477028992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787830041.803666:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830041.805202:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830041.805207:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830041.805210:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830041.805220:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4409 00000100:00080000:0.0:1787830041.805229:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830041.805237:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830041.805260:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830041.805291:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477028992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830041.805292:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477029120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830041.805319:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830041.805323:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830041.805325:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830041.805329:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4409 00000800:00000100:2.0:1787830041.805350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830041.805351:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830041.805359:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830041.805365:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830041.805393:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477029248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830041.805398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830041/real 1787830041] req@ffff9a7890a3da40 x1874671477028992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830041.805410:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830041.805415:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830041.805417:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830041.805422:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4409 00000100:00100000:2.0:1787830041.805425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477028992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830041.805427:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830041.805436:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:2.0:1787830041.805445:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:0.0:1787830041.805446:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:2.0:1787830041.805454:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830041.805459:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477028992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830041.805482:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830041.805486:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477029376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830041.805510:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1787830041.805511:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830041.805518:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830041.805520:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830041.805525:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4409 00000100:00080000:0.0:1787830041.805530:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787830041.805530:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830041/real 1787830041] req@ffff9a7890a3f480 x1874671477029120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830041.805540:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830041.805549:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830041.805555:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477029120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830041.805566:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477029504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830041.805575:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830041.805579:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830041.805582:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830041.805595:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830041.805631:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830041.805644:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830041/real 1787830041] req@ffff9a7890a3de00 x1874671477029248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830041.805656:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477029248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830041.805675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830041.805679:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830041.805682:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830041.805694:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830041.805714:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830041.805725:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830041/real 1787830041] req@ffff9a7890a3c000 x1874671477029376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830041.805736:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477029376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830041.805744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830041.805747:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830041.805937:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830041.805944:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830041.805966:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830041.805978:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830041/real 1787830041] req@ffff9a7890a3fc00 x1874671477029504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830041.805989:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477029504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830041.805999:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830041.806002:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830041.806005:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830042.826065:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830043.850394:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830044.874040:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830045.898067:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830046.922760:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830046.922769:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830046.922775:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830046.922785:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4414 00000100:00080000:1.0:1787830046.922797:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830046.922804:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830046.922876:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477029632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830046.922907:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830046.922911:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830046.922912:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830046.922916:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4414 00000100:00080000:1.0:1787830046.922919:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830046.922924:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830046.922934:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787830046.922949:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830046.922950:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477029760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830046.922966:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830046.922969:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830046.922972:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830046.922976:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4414 00000100:00080000:1.0:1787830046.922978:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830046.922982:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830046.922986:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830046.923008:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477029888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830046.923021:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787830046.923028:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477029632:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830046.923029:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830046.923035:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830046.923051:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4414 00000100:00080000:1.0:1787830046.923056:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830046.923059:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830046.923065:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787830046.923120:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830046.923193:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830046/real 1787830046] req@ffff9a7890b1c000 x1874671477029632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830046.923213:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477029632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830046.923225:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830046.923229:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830046.923233:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477029632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830046.923242:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830046.923262:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830046.923274:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830046/real 1787830046] req@ffff9a7890b1f840 x1874671477029760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830046.923288:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477029760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830046.923296:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830046.923299:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830046.923302:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830046.923333:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830046.923350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830046.923360:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830046/real 1787830046] req@ffff9a7890b1f0c0 x1874671477029888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830046.923373:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477029888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830046.923381:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830046.923384:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830046.923388:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477029888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830046.924069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477030016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830046.924103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830046.924107:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830046.924109:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830046.924114:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4414 00000100:00080000:1.0:1787830046.924119:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830046.924123:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830046.924130:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830046.924142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477030144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830046.926098:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830046.926173:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830046.926205:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830046/real 1787830046] req@ffff9a7890b1fc00 x1874671477030016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830046.926223:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477030016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830046.926236:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830046.926242:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830046.926246:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830046.926253:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830046.926271:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830046.926281:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830046/real 1787830046] req@ffff9a7890b1f480 x1874671477030144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830046.926292:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477030144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830071 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830046.926309:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830046.926312:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830046.926315:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830047.945980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830048.970106:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830049.994100:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830051.018008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830052.042069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830052.042298:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830052.042306:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830052.042311:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.042323:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4419 00000100:00080000:0.0:1787830052.042336:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.042345:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830052.042392:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477030272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830052.042420:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830052.042423:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830052.042425:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.042429:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4419 00000100:00080000:0.0:1787830052.042433:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.042438:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830052.042450:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830052.042464:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477030272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830052.042469:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477030400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830052.042482:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830052.042484:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830052.042487:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.042492:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4419 00000100:00080000:0.0:1787830052.042495:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.042500:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830052.042506:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830052.042518:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477030528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830052.042528:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830052.042531:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830052.042533:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.042537:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4419 00000100:00080000:0.0:1787830052.042540:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.042544:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830052.042549:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830052.042560:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477030656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830052.042568:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830052.042571:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830052.042573:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830052.042577:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4419 00000100:00080000:0.0:1787830052.042580:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830052.042583:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830052.042588:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830052.042600:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477030784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830052.042645:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830052.042677:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830052.042701:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830052.042725:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830053.066767:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830054.090006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830055.114077:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787830055.118818:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787830055.118882:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830055.118897:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830055.118949:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830055.118960:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830055.118967:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830055.118974:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787830055.119080:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830055] req@ffff9a7890a3fc00 x1874671477030400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830055.119104:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477030400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830055.119116:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830055.119121:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830055.119125:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830055.119132:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830055] req@ffff9a7890a3c000 x1874671477030528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830055.119151:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477030528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830055.119160:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830055.119163:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830055.119166:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830055.119173:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830055] req@ffff9a7890a3de00 x1874671477030656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830055.119185:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477030656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830055.119193:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830055.119196:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830055.119199:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830055.119205:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830055] req@ffff9a7890a3f480 x1874671477030784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830055.119292:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477030784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830055.119302:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830055.119305:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830055.119308:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477030784:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830055.119327:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830052/real 1787830055] req@ffff9a7890a3c3c0 x1874671477030272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830055.119339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477030272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830055.119346:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830055.119349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830055.119352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477030272:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787830056.138180:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830057.162043:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830057.162045:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830057.162052:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830057.162057:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830057.162067:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4424 00000100:00080000:0.0:1787830057.162077:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830057.162084:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830057.162119:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477030912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830057.162188:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477030912:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830057.162244:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830057.162273:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830057.162278:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830057.162280:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830057.162284:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4424 00000100:00080000:0.0:1787830057.162287:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830057.162291:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830057.162300:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830057.162314:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477031040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830057.162500:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff9a7890a3da40 x1874671477030912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830057.162516:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477030912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830057.162526:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830057.162530:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830057.162534:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477030912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830057.162556:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830057.162578:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830057.162590:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff9a7890a3c3c0 x1874671477031040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830057.162602:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477031040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830057.162615:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830057.162618:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830057.162621:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031040:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830057.166351:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830057.166361:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830057.166365:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830057.166384:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4424 00000100:00080000:0.0:1787830057.166393:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830057.166401:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830057.166411:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830057.166459:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477031168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830057.166473:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830057.166477:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830057.166478:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830057.166482:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4424 00000100:00080000:0.0:1787830057.166486:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830057.166490:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830057.166495:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830057.166518:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477031296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830057.166528:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830057.166531:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830057.166532:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830057.166537:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4424 00000100:00080000:0.0:1787830057.166540:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830057.166543:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830057.166547:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830057.166558:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477031424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830057.167331:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830057.167551:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830057.167580:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff9a7890a3f480 x1874671477031168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830057.167596:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477031168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830057.167607:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830057.167611:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830057.167615:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830057.167622:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830057.167644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830057.167997:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff9a7890a3de00 x1874671477031296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830057.168015:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477031296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830057.168024:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830057.168027:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830057.168030:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830057.168036:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830057.168070:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830057.168083:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830057/real 1787830057] req@ffff9a7890a3c000 x1874671477031424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830057.168095:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477031424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830057.168103:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830057.168106:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830057.168109:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830058.186089:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830059.210303:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830060.233948:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830061.259279:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830062.282058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830062.283553:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830062.283559:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830062.283564:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830062.283574:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4429 00000100:00080000:2.0:1787830062.283584:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830062.283590:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830062.283625:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477031552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830062.283699:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477031552:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830062.283758:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830062.283789:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830062.283793:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830062.283795:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830062.283799:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4429 00000100:00080000:2.0:1787830062.283803:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830062.283807:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830062.283817:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830062.283846:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477031680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830062.283965:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9a78841dbc00 x1874671477031552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830062.283984:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477031552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830062.283994:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830062.283998:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830062.284003:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477031552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830062.284024:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830062.284050:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830062.284062:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9a78841db0c0 x1874671477031680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830062.284075:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477031680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830062.284083:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830062.284087:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830062.284090:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031680:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830062.284564:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830062.284569:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830062.284571:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830062.284576:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4429 00000100:00080000:2.0:1787830062.284581:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830062.284585:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830062.284590:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830062.284604:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477031808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830062.284614:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830062.284617:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830062.284619:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830062.284622:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4429 00000100:00080000:2.0:1787830062.284625:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830062.284629:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830062.284634:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830062.284644:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477031936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830062.284653:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830062.284656:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830062.284658:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830062.284661:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4429 00000100:00080000:2.0:1787830062.284664:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830062.284668:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830062.284672:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830062.284683:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477032064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830062.284747:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830062.284771:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830062.284784:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9a78841da1c0 x1874671477031808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830062.284797:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477031808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830062.284805:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830062.284808:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830062.284811:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830062.284817:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830062.284856:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830062.284943:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9a78841d9a40 x1874671477031936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830062.284963:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477031936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830062.284972:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830062.284975:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830062.284978:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477031936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830062.284984:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830062.285005:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830062.285017:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830062/real 1787830062] req@ffff9a78841d8780 x1874671477032064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830062.285029:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477032064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830062.285038:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830062.285041:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830062.285044:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830063.305964:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830064.330078:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830065.357205:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830066.378038:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830067.402121:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830067.402187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830067.402195:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830067.402201:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830067.402213:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4434 00000100:00080000:0.0:1787830067.402226:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830067.402234:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830067.402283:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477032192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830067.402315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830067.402318:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830067.402321:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830067.402325:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4434 00000100:00080000:0.0:1787830067.402329:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830067.402333:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830067.402344:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830067.402359:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477032320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830067.402369:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830067.402373:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830067.402375:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830067.402380:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4434 00000100:00080000:0.0:1787830067.402383:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830067.402387:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830067.402392:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830067.402404:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477032448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830067.402412:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830067.402415:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830067.402417:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830067.402421:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4434 00000100:00080000:0.0:1787830067.402424:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830067.402428:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830067.402433:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830067.402441:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477032576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830067.402449:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830067.402452:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830067.402454:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830067.402457:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4434 00000100:00080000:0.0:1787830067.402460:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830067.402463:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830067.402467:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830067.402478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477032704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830067.402717:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477032192:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830067.402795:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830067.402878:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9a7890a3fc00 x1874671477032192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830067.402896:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477032192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830067.402908:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830067.402912:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830067.402916:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477032192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830067.402923:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830067.402944:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830067.402955:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9a7890a3c000 x1874671477032320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830067.402967:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477032320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830067.402975:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830067.402978:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830067.402982:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830067.402987:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830067.403005:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830067.403015:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9a7890a3de00 x1874671477032448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830067.403025:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477032448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830067.403033:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830067.403036:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830067.403040:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830067.403046:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830067.403062:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830067.403072:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9a7890a3f480 x1874671477032576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830067.403083:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477032576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830067.403092:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830067.403095:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830067.403098:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830067.403103:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830067.403116:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830067.403124:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830067/real 1787830067] req@ffff9a7890a3c3c0 x1874671477032704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830067.403134:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477032704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830067.403143:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830067.403145:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830067.403148:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830068.427921:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830069.450370:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830070.474047:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830071.498156:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830072.522197:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830072.522223:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830072.522228:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830072.522259:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4440 00000100:00080000:1.0:1787830072.522278:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830072.522285:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787830072.522322:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830072.522335:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477032832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830072.522391:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830072.522396:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830072.522397:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830072.522401:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4440 00000100:00080000:1.0:1787830072.522405:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830072.522411:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830072.522421:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830072.522430:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477032832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830072.522436:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477032960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830072.522448:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830072.522451:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830072.522453:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830072.522457:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4440 00000100:00080000:1.0:1787830072.522461:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830072.522465:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830072.522470:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830072.522481:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477033088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830072.522490:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830072.522494:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830072.522495:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830072.522501:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4440 00000800:00000100:3.0:1787830072.522503:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830072.522505:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830072.522510:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830072.522516:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830072.522527:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477033216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830072.522536:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830072.522539:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830072.522541:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830072.522544:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4440 00000100:00080000:1.0:1787830072.522547:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830072.522551:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830072.522561:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830072.522571:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477033344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830072.522619:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830072/real 1787830072] req@ffff9a7890b1da40 x1874671477032832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830072.522638:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477032832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830072.522647:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830072.522651:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830072.522655:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477032832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830072.522676:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830072.522698:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830072.522710:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830072/real 1787830072] req@ffff9a7890b1d680 x1874671477032960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830072.522721:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477032960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830072.522728:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830072.522731:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830072.522734:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477032960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830072.522740:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830072.522757:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830072.522768:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830072/real 1787830072] req@ffff9a7890b1de00 x1874671477033088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830072.522783:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477033088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830072.522791:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830072.522794:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830072.522796:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830072.522802:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830072.522849:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830072.522861:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830072/real 1787830072] req@ffff9a7890b1e580 x1874671477033216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830072.522872:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477033216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830072.522881:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830072.522884:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830072.522887:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830072.522891:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830072.522907:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830072.522917:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830072/real 1787830072] req@ffff9a7890b1e940 x1874671477033344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830072.522926:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477033344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830072.522939:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830072.522942:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830072.522946:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830073.546377:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830074.578641:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830075.594160:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830076.618035:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830077.641965:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830077.641974:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830077.641978:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830077.641988:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4445 00000100:00080000:0.0:1787830077.642001:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830077.642008:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787830077.642030:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830077.642042:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477033472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830077.642065:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830077.642068:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830077.642070:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830077.642074:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4445 00000100:00080000:0.0:1787830077.642078:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830077.642081:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830077.642090:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830077.642112:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477033600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830077.642116:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477033472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830077.642122:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830077.642125:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830077.642126:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830077.642130:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4445 00000100:00080000:0.0:1787830077.642133:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830077.642136:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830077.642140:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830077.642149:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477033728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830077.642157:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830077.642159:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830077.642161:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830077.642164:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4445 00000100:00080000:0.0:1787830077.642166:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830077.642170:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787830077.642170:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787830077.642174:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830077.642181:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477033856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830077.642188:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830077.642190:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830077.642192:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830077.642195:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4445 00000100:00080000:0.0:1787830077.642197:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830077.642199:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830077.642203:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830077.642211:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477033984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830077.642211:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830077/real 1787830077] req@ffff9a7890a3da40 x1874671477033472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830077.642334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477033472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830077.642345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830077.642349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830077.642353:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477033472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830077.642373:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830077.642395:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830077.642407:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830077/real 1787830077] req@ffff9a7890a3c3c0 x1874671477033600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830077.642418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477033600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830077.642426:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830077.642429:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830077.642433:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830077.642439:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830077.642458:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830077.642468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830077/real 1787830077] req@ffff9a7890a3f480 x1874671477033728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830077.642479:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477033728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830077.642486:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830077.642489:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830077.642493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830077.642498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830077.642514:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830077.642525:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830077/real 1787830077] req@ffff9a7890a3de00 x1874671477033856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830077.642536:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477033856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830077.642544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830077.642547:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830077.642550:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830077.642556:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830077.642573:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830077.642584:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830077/real 1787830077] req@ffff9a7890a3c000 x1874671477033984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830077.642595:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477033984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830077.642604:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830077.642607:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830077.642610:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477033984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830078.666667:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830079.690152:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830080.714109:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830081.738157:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830082.761982:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830082.761993:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830082.761997:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830082.762007:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4450 00000100:00080000:0.0:1787830082.762027:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787830082.762041:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830082.762044:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830082.762078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477034112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830082.762108:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830082.762112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830082.762114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830082.762117:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4450 00000100:00080000:0.0:1787830082.762121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830082.762125:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830082.762134:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830082.762147:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477034240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830082.762157:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830082.762159:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830082.762161:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830082.762164:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4450 00000100:00080000:0.0:1787830082.762168:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830082.762172:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830082.762176:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830082.762186:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477034368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830082.762194:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830082.762197:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830082.762199:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830082.762202:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4450 00000100:00080000:0.0:1787830082.762205:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830082.762208:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830082.762212:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830082.762221:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477034496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830082.762229:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830082.762232:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830082.762233:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830082.762237:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4450 00000100:00080000:0.0:1787830082.762239:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830082.762243:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830082.762246:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830082.762256:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477034624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830082.762334:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477034112:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830082.762429:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830082.762472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830082/real 1787830082] req@ffff9a7890a3fc00 x1874671477034112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830082.762500:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477034112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830082.762509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830082.762512:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830082.762515:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477034112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830082.762521:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830082.762541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830082.762553:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830082/real 1787830082] req@ffff9a7890a3c000 x1874671477034240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830082.762567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477034240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830082.762573:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830082.762575:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830082.762578:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830082.762582:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830082.762598:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830082.762607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830082/real 1787830082] req@ffff9a7890a3de00 x1874671477034368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830082.762618:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477034368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830082.762626:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830082.762629:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830082.762632:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830082.762637:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830082.762654:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830082.762662:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830082/real 1787830082] req@ffff9a7890a3f480 x1874671477034496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830082.762670:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477034496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830082.762683:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830082.762686:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830082.762689:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830082.762694:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830082.762710:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830082.762720:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830082/real 1787830082] req@ffff9a7890a3c3c0 x1874671477034624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830082.762730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477034624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830082.762739:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830082.762742:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830082.762745:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830083.786357:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830084.810016:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830085.834503:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830086.861324:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830087.882027:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830087.882444:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830087.882451:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830087.882455:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830087.882464:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4455 00000100:00080000:2.0:1787830087.882474:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830087.882481:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830087.882516:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477034752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830087.882543:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830087.882547:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830087.882549:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830087.882552:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4455 00000100:00080000:2.0:1787830087.882555:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830087.882559:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830087.882568:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830087.882578:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477034880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830087.882586:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830087.882589:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830087.882590:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830087.882593:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4455 00000100:00080000:2.0:1787830087.882596:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830087.882599:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830087.882602:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830087.882611:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477035008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830087.882618:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830087.882622:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830087.882624:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830087.882627:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4455 00000100:00080000:2.0:1787830087.882630:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830087.882633:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830087.882637:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830087.882647:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477035136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830087.882656:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830087.882659:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830087.882660:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830087.882663:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4455 00000100:00080000:2.0:1787830087.882666:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830087.882669:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830087.882673:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830087.882683:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477035264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830087.883216:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477034752:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830087.883274:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830087.883321:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830087/real 1787830087] req@ffff9a78841db480 x1874671477034752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830087.883335:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477034752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830087.883344:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830087.883349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830087.883352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477034752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830087.883359:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830087.883378:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830087.883389:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830087/real 1787830087] req@ffff9a78841da580 x1874671477034880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830087.883400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477034880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830087.883406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830087.883409:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830087.883412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477034880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830087.883417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830087.883435:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830087.883446:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830087/real 1787830087] req@ffff9a78841d8000 x1874671477035008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830087.883457:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477035008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830087.883464:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830087.883467:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830087.883471:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830087.883476:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830087.883494:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830087.883506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830087/real 1787830087] req@ffff9a78841d83c0 x1874671477035136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830087.883515:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477035136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830087.883524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830087.883527:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830087.883530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830087.883535:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830087.883551:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830087.883560:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830087/real 1787830087] req@ffff9a78841d9e00 x1874671477035264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830087.883569:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477035264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830087.883577:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830087.883580:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830087.883583:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830088.906071:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830089.930063:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830090.956667:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830091.978042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830093.002121:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830093.002370:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830093.002376:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830093.002380:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830093.002390:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4460 00000100:00080000:0.0:1787830093.002401:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830093.002407:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830093.002443:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477035392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830093.002478:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830093.002737:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830093.002739:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830093.002744:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4460 00000100:00080000:0.0:1787830093.002748:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830093.002752:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830093.002762:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830093.002777:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477035520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830093.002786:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830093.002789:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830093.002791:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830093.002794:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4460 00000100:00080000:0.0:1787830093.002797:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830093.002802:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830093.002806:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830093.002816:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477035648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830093.002824:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830093.002827:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830093.002861:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830093.002866:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4460 00000100:00080000:0.0:1787830093.002869:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830093.002873:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830093.002878:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830093.002888:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477035776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830093.002895:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830093.002898:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830093.002899:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830093.002903:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4460 00000100:00080000:0.0:1787830093.002905:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830093.002908:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830093.002912:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830093.002920:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477035904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830093.003033:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477035392:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830093.003087:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830093.003128:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9a7890a3da40 x1874671477035392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830093.003149:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477035392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830093.003158:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830093.003161:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830093.003165:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477035392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830093.003172:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830093.003189:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830093.003199:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9a7890a3c3c0 x1874671477035520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830093.003208:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477035520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830093.003214:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830093.003217:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830093.003219:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830093.003225:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830093.003240:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830093.003249:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9a7890a3f480 x1874671477035648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830093.003258:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477035648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830093.003269:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830093.003272:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830093.003275:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830093.003538:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830093.003563:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830093.003576:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9a7890a3de00 x1874671477035776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830093.003586:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477035776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830093.003593:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830093.003597:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830093.003599:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830093.003604:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830093.003619:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830093.003629:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830093/real 1787830093] req@ffff9a7890a3c000 x1874671477035904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830093.003639:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477035904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830093.003647:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830093.003650:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830093.003653:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477035904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830094.026003:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830095.050118:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830096.075240:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830097.098804:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830098.122008:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830098.122019:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830098.122024:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830098.122033:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4465 00000100:00080000:0.0:1787830098.122044:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830098.122051:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830098.122085:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477036032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830098.122112:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830098.122115:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830098.122117:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830098.122121:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4465 00000100:00080000:0.0:1787830098.122125:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830098.122128:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830098.122137:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830098.122150:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477036160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830098.122161:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830098.122164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830098.122166:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830098.122170:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4465 00000100:00080000:0.0:1787830098.122174:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787830098.122177:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830098.122179:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830098.122186:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830098.122198:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477036288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830098.122212:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830098.122215:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830098.122217:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830098.122221:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4465 00000100:00080000:0.0:1787830098.122224:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830098.122227:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830098.122231:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830098.122241:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477036416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830098.122249:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830098.122252:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830098.122253:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830098.122257:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4465 00000100:00080000:0.0:1787830098.122259:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830098.122263:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830098.122272:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830098.122282:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477036544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830098.122317:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477036032:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830098.122392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830098.122487:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9a7890a3fc00 x1874671477036032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830098.122500:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477036032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830098.122507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830098.122509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830098.122512:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477036032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830098.122517:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830098.122533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830098.122544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9a7890a3c000 x1874671477036160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830098.122555:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477036160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830098.122562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830098.122565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830098.122568:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830098.122574:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830098.122593:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830098.122605:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9a7890a3de00 x1874671477036288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830098.122616:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477036288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830098.122624:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830098.122627:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830098.122630:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830098.122636:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830098.122668:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830098.122680:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9a7890a3f480 x1874671477036416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830098.122690:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477036416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830098.122704:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830098.122707:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830098.122710:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830098.122715:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830098.122733:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830098.122746:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830098/real 1787830098] req@ffff9a7890a3c3c0 x1874671477036544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830098.122762:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477036544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830098.122780:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830098.122783:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830098.122786:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830099.146217:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830100.170020:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830101.195500:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830102.218090:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830103.242051:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830103.242092:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830103.242101:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830103.242106:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830103.242119:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4470 00000100:00080000:2.0:1787830103.242133:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830103.242142:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830103.242193:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477036672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830103.242222:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830103.242226:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830103.242228:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830103.242231:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4470 00000100:00080000:2.0:1787830103.242234:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830103.242238:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830103.242250:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830103.242263:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477036800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830103.242272:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830103.242275:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830103.242277:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830103.242281:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4470 00000100:00080000:2.0:1787830103.242284:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830103.242288:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830103.242292:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830103.242302:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477036928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830103.242315:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830103.242318:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830103.242320:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830103.242324:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4470 00000100:00080000:2.0:1787830103.242327:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830103.242330:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830103.242335:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830103.242430:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477036672:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830103.242509:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830103.242570:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff9a78841d92c0 x1874671477036672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830103.242592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477036672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830103.242613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830103.242618:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830103.242623:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477036672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830103.242631:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830103.242655:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830103.242668:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff9a78841d9680 x1874671477036800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830103.242680:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477036800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830103.242688:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830103.242692:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830103.242695:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830103.242700:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830103.242717:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830103.242728:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff9a78841d8b40 x1874671477036928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830103.242738:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477036928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830103.242746:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830103.242749:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830103.242751:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477036928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830103.243536:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477037056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830103.243572:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830103.243577:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830103.243579:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830103.243585:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4470 00000100:00080000:2.0:1787830103.243591:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830103.243596:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830103.243617:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830103.243642:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477037184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830103.243657:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830103.243698:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830103.243717:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff9a78841da940 x1874671477037056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830103.243730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477037056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830103.243742:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830103.243746:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830103.243750:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830103.243766:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830103.243786:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830103.243798:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830103/real 1787830103] req@ffff9a78841db840 x1874671477037184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830103.243808:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477037184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830103.243817:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830103.243820:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830103.243823:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830104.266051:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830105.290126:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830106.314008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830107.338055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830108.362633:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.362644:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830108.362648:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.362658:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4475 00000100:00080000:0.0:1787830108.362670:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.362678:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830108.362713:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477037312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830108.362750:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.362754:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:1.0:1787830108.362810:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477037312:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830108.362890:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830108.362935:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff9a7890a3da40 x1874671477037312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830108.362955:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477037312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830108.362967:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830108.362971:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830108.362976:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477037312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830108.363055:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.363060:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4475 00000100:00080000:0.0:1787830108.363064:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.363068:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830108.363077:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830108.363089:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477037440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830108.363109:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.363112:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830108.363114:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.363118:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4475 00000100:00080000:0.0:1787830108.363121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.363125:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830108.363129:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830108.363139:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477037568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830108.363147:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.363150:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830108.363152:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.363155:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4475 00000100:00080000:0.0:1787830108.363158:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.363161:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830108.363166:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830108.363174:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477037696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830108.363181:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830108.363184:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830108.363186:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830108.363189:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4475 00000100:00080000:0.0:1787830108.363192:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830108.363195:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830108.363199:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830108.363207:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477037824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830108.363263:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830108.363287:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830108.363299:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff9a7890a3da40 x1874671477037440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830108.363310:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477037440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830108.363318:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830108.363320:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830108.363324:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830108.363329:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830108.363345:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830108.363355:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff9a7890a3c3c0 x1874671477037568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830108.363366:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477037568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830108.363373:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830108.363376:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830108.363379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830108.363384:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830108.363400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830108.363410:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff9a7890a3f480 x1874671477037696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830108.363421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477037696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830108.363430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830108.363432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830108.363435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830108.363440:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830108.363457:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830108.363468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830108/real 1787830108] req@ffff9a7890a3de00 x1874671477037824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830108.363478:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477037824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830108.363486:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830108.363489:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830108.363492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477037824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830108.363665:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830109.386069:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830110.413232:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830111.434061:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830112.458098:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830113.482047:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830113.482057:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830113.482061:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830113.482072:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4480 00000100:00080000:0.0:1787830113.482082:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830113.482090:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830113.482128:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477037952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830113.482189:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477037952:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830113.482234:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:3.0:1787830113.482243:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830113.482275:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830113.482278:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830113.482281:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830113.482285:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4480 00000100:00080000:0.0:1787830113.482289:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830113.482293:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830113.482303:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830113.482314:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477038080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830113.482378:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830113/real 1787830113] req@ffff9a7890a3c000 x1874671477037952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830113.482394:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477037952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830113.482404:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830113.482408:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830113.482412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477037952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830113.482435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830113.482457:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830113.482469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830113/real 1787830113] req@ffff9a7890a3fc00 x1874671477038080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830113.482480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477038080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830113.482488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830113.482490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830113.482493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830113.482592:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830113.482596:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830113.482598:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830113.482602:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4480 00000100:00080000:0.0:1787830113.482606:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830113.482609:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830113.482614:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830113.482627:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477038208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830113.482670:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830113.482692:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830113.482713:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830113.482716:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830113.482718:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830113.482721:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4480 00000100:00080000:0.0:1787830113.482725:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830113.482728:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830113.482733:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830113.482744:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477038336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830113.482780:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830113/real 1787830113] req@ffff9a7890a3fc00 x1874671477038208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830113.482792:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477038208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830113.482800:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830113.482803:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830113.482806:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830113.482816:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830113.482851:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830113.482862:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830113/real 1787830113] req@ffff9a7890a3c000 x1874671477038336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830113.482873:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477038336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830113.482881:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830113.482884:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830113.482887:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038336:192.168.203.140@tcp:8:kworker.0 00000100:00080000:0.0:1787830113.482945:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830113.482950:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830113.482952:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830113.482956:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4480 00000100:00080000:0.0:1787830113.482961:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830113.482965:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830113.482971:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830113.482982:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477038464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830113.483134:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830113.483162:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830113.483174:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830113/real 1787830113] req@ffff9a7890a3de00 x1874671477038464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830113.483188:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477038464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830113.483199:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830113.483203:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830113.483207:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830114.506012:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830115.530038:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830116.554066:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830117.579141:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830118.601976:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830118.601991:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830118.601996:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830118.602008:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4486 00000100:00080000:2.0:1787830118.602022:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787830118.602026:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830118.602028:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830118.602081:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477038592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830118.602109:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830118.602113:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830118.602115:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830118.602118:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4486 00000100:00080000:2.0:1787830118.602122:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830118.602127:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830118.602138:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830118.602149:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477038592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830118.602154:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477038720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830118.602165:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830118.602169:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830118.602171:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830118.602175:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4486 00000100:00080000:2.0:1787830118.602179:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830118.602183:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830118.602188:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830118.602200:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477038848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830118.602206:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830118.602209:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830118.602210:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830118.602214:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4486 00000100:00080000:2.0:1787830118.602218:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830118.602221:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830118.602226:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830118.602238:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477038976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830118.602247:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830118.602251:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830118.602253:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830118.602258:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4486 00000100:00080000:2.0:1787830118.602262:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830118.602266:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830118.602269:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830118.602280:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477039104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830118.602281:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830118.602320:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830118.602344:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830118.602359:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830119.626215:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830120.651089:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830121.674055:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787830121.674257:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787830121.674307:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787830121.674328:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830121.674375:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830121.674388:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830121.674399:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830121.674407:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787830121.674422:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830118/real 1787830121] req@ffff9a78841da940 x1874671477038720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830121.674448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477038720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830121.674468:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830121.674473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830121.674490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830121.674499:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830118/real 1787830121] req@ffff9a78841d8b40 x1874671477038848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830121.674511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477038848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830121.674518:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830121.674522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830121.674525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830121.674531:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830118/real 1787830121] req@ffff9a78841d9680 x1874671477038976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830121.674542:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477038976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830121.674552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830121.674555:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830121.674559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477038976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830121.674566:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830118/real 1787830121] req@ffff9a78841db840 x1874671477038592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830121.674578:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477038592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830121.674586:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830121.674589:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830121.674592:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477038592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830121.674619:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830118/real 1787830121] req@ffff9a78841d92c0 x1874671477039104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830121.674640:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477039104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830121.674649:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830121.674653:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830121.674655:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830122.698368:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830123.721961:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830123.721971:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830123.721976:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830123.721987:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4491 00000100:00080000:2.0:1787830123.721997:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830123.722004:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830123.722051:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477039232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830123.722078:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830123.722082:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830123.722084:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830123.722088:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4491 00000100:00080000:2.0:1787830123.722100:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830123.722104:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830123.722112:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830123.722123:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477039360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830123.722133:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830123.722136:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830123.722138:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830123.722142:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4491 00000100:00080000:2.0:1787830123.722145:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830123.722153:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477039232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830123.722162:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830123.722169:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830123.722182:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477039488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830123.722197:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830123.722201:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830123.722203:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787830123.722204:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830123.722209:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4491 00000100:00080000:2.0:1787830123.722215:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787830123.722219:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830123.722223:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830123.722229:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830123.722242:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477039616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830123.722251:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830123.722255:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787830123.722267:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830123/real 1787830123] req@ffff9a78841d9e00 x1874671477039232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830123.722269:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830123.722277:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4491 00000100:00080000:2.0:1787830123.722284:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830123.722289:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830123.722294:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830123.722297:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477039232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830123.722309:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477039744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830123.722310:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830123.722319:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830123.722327:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477039232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830123.722337:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830123.722360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830123.722372:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830123/real 1787830123] req@ffff9a78841d83c0 x1874671477039360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830123.722384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477039360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830123.722393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830123.722396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830123.722399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830123.722420:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830123.722438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830123.722448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830123/real 1787830123] req@ffff9a78841d8000 x1874671477039488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830123.722459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477039488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830123.723691:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830123.723699:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830123.723704:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830123.723713:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830123.723758:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830123.723779:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830123/real 1787830123] req@ffff9a78841da580 x1874671477039616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830123.723790:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477039616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830123.723799:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830123.723802:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830123.723805:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830123.723811:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830123.723845:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830123.723857:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830123/real 1787830123] req@ffff9a78841db480 x1874671477039744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830123.723868:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477039744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830123.723876:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830123.723878:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830123.723881:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477039744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830124.746098:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830125.770061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830126.794656:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830127.818995:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830128.841986:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830128.841996:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830128.842001:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830128.842012:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4496 00000100:00080000:2.0:1787830128.842024:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830128.842031:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830128.842072:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477039872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830128.842092:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830128.842101:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830128.842105:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830128.842107:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830128.842110:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4496 00000100:00080000:2.0:1787830128.842113:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830128.842116:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830128.842124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830128.842132:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477040000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830128.842138:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830128.842140:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830128.842141:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830128.842144:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4496 00000100:00080000:2.0:1787830128.842145:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830128.842147:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830128.842149:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830128.842155:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477040128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830128.842160:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830128.842163:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830128.842164:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830128.842166:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4496 00000100:00080000:2.0:1787830128.842168:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830128.842169:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830128.842172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830128.842177:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477040256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830128.842182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830128.842183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830128.842184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830128.842186:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4496 00000100:00080000:2.0:1787830128.842188:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830128.842190:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830128.842192:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830128.842200:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477040384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830128.842279:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477039872:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830128.842341:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830128.842384:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830128/real 1787830128] req@ffff9a78841d8780 x1874671477039872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830128.842396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477039872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830128.842404:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830128.842407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830128.842410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477039872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830128.842414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830128.842443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830128.842455:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830128/real 1787830128] req@ffff9a78841d9a40 x1874671477040000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830128.842467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477040000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830128.842475:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830128.842477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830128.842480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830128.842483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830128.842499:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830128.842506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830128/real 1787830128] req@ffff9a78841da1c0 x1874671477040128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830128.842512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477040128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830128.842517:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830128.842518:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830128.842520:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830128.842523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830128.842533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830128.842539:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830128/real 1787830128] req@ffff9a78841db0c0 x1874671477040256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830128.842545:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477040256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830128.842551:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830128.842553:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830128.842554:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830128.842557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830128.842566:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830128.842570:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830128/real 1787830128] req@ffff9a78841dbc00 x1874671477040384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830128.842581:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477040384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830128.842587:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830128.842589:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830128.842590:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830129.866912:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830130.890462:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830131.914450:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830132.937990:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830133.961986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830133.962008:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830133.962015:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830133.962029:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830133.962040:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4501 00000100:00080000:1.0:1787830133.962049:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830133.962064:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830133.962103:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477040512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830133.962134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830133.962138:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830133.962140:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830133.962144:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4501 00000100:00080000:1.0:1787830133.962148:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830133.962152:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830133.962162:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830133.962174:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477040512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830133.962177:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477040640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830133.962201:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830133.962204:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830133.962206:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830133.962210:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4501 00000100:00080000:1.0:1787830133.962222:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830133.962229:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787830133.962230:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787830133.962236:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830133.962247:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477040768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830133.962254:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830133.962257:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830133.962258:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830133.962262:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4501 00000100:00080000:1.0:1787830133.962266:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830133.962269:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830133.962274:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830133.962289:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477040896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830133.962298:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830133.962301:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830133.962304:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830133.962308:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4501 00000100:00080000:1.0:1787830133.962311:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830133.962314:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830133.962318:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830133.962331:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477041024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830133.962340:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830133/real 1787830133] req@ffff9a7890b1e940 x1874671477040512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830133.962360:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477040512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830133.962371:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830133.962375:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830133.962379:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477040512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830133.962414:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830133.962435:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830133.962447:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830133/real 1787830133] req@ffff9a7890b1e580 x1874671477040640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830133.962458:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477040640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830133.962467:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830133.962470:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830133.962473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830133.962479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830133.962492:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830133.962501:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830133/real 1787830133] req@ffff9a7890b1de00 x1874671477040768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830133.962511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477040768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830133.962519:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830133.962522:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830133.962525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830133.962530:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830133.962544:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830133.962552:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830133/real 1787830133] req@ffff9a7890b1d680 x1874671477040896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830133.962562:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477040896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830133.962570:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830133.962573:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830133.962576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477040896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830133.962582:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830133.962607:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830133.962616:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830133/real 1787830133] req@ffff9a7890b1da40 x1874671477041024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830133.962626:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477041024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830158 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830133.962635:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830133.962638:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830133.962641:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830134.986497:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830136.010127:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830137.034179:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830138.058104:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830139.082271:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830139.084676:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830139.084689:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830139.084695:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830139.084708:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4506 00000100:00080000:2.0:1787830139.084722:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830139.084731:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830139.084783:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477041152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830139.084819:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830139.084823:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830139.084824:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830139.084828:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4506 00000100:00080000:2.0:1787830139.084877:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830139.084883:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830139.084894:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830139.084899:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477041152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830139.084913:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477041280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830139.084926:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830139.084931:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830139.084932:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830139.084937:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4506 00000100:00080000:2.0:1787830139.084941:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830139.084945:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830139.084950:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830139.084964:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477041408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830139.084973:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830139.084976:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830139.084978:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830139.084982:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4506 00000100:00080000:2.0:1787830139.084988:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787830139.084995:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830139.084999:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830139.085008:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830139.085024:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477041536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830139.085034:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830139.085037:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830139.085039:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830139.085042:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4506 00000100:00080000:2.0:1787830139.085046:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830139.085050:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830139.085055:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830139.085066:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477041664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830139.085130:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830139] req@ffff9a78841dbc00 x1874671477041152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830139.085149:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477041152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830139.085159:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830139.085163:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830139.085167:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477041152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830139.085190:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830139.085222:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830139.085234:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830139] req@ffff9a78841db0c0 x1874671477041280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830139.085246:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477041280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830139.085253:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830139.085257:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830139.085260:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830139.085267:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830139.085293:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830139.085304:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830139] req@ffff9a78841da1c0 x1874671477041408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830139.085314:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477041408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830139.085322:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830139.085325:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830139.085328:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830139.085334:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830139.085352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830139.085376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830139] req@ffff9a78841d9a40 x1874671477041536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830139.085394:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477041536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830139.085403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830139.085416:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830139.085420:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830139.085426:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830139.085451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830139.085461:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830139/real 1787830139] req@ffff9a78841d8780 x1874671477041664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830139.085472:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477041664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830139.085481:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830139.085484:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830139.085487:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830140.106562:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830141.130139:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830142.154009:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830143.178180:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830144.202069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830144.202371:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830144.202380:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830144.202385:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830144.202398:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4511 00000100:00080000:1.0:1787830144.202412:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830144.202420:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830144.202470:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477041792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830144.202506:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830144.202509:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830144.202512:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830144.202516:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4511 00000100:00080000:1.0:1787830144.202520:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830144.202525:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830144.202536:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830144.202550:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477041920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830144.202560:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830144.202564:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830144.202566:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830144.202570:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4511 00000100:00080000:1.0:1787830144.202574:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830144.202578:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830144.202583:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830144.202585:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477041792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830144.202598:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477042048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830144.202609:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830144.202612:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830144.202614:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787830144.202673:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830144.202728:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff9a7890b1f480 x1874671477041792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830144.202757:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477041792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830144.202770:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830144.202778:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830144.202782:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477041792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830144.202790:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830144.202970:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830144.202989:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff9a7890b1fc00 x1874671477041920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830144.203002:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477041920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830144.203010:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830144.203014:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830144.203017:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477041920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830144.203060:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830144.203081:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830144.203092:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff9a7890b1f0c0 x1874671477042048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830144.203108:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477042048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830144.203116:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830144.203119:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830144.203123:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042048:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830144.204417:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4511 00000100:00080000:1.0:1787830144.204427:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830144.204434:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830144.204445:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830144.204475:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477042176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830144.204510:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830144.204514:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830144.204516:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830144.204521:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4511 00000100:00080000:1.0:1787830144.204525:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830144.204530:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830144.204535:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830144.204547:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477042304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830144.204556:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830144.204623:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830144.204648:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff9a7890b1f840 x1874671477042176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830144.204663:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477042176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830144.204675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830144.204679:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830144.204683:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830144.204704:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830144.204723:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830144.204734:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830144/real 1787830144] req@ffff9a7890b1c000 x1874671477042304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830144.204747:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477042304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830144.204756:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830144.204759:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830144.204762:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830145.226115:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830146.250022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830147.274180:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830148.298188:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830149.323579:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830149.324651:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830149.324660:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830149.324665:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830149.324678:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4516 00000100:00080000:0.0:1787830149.324691:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830149.324698:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830149.324748:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477042432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830149.324816:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477042432:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830149.324909:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830149.324975:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff9a7890a3f480 x1874671477042432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830149.324996:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477042432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830149.325009:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830149.325013:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830149.325018:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477042432:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830149.325022:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830149.325028:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830149.325031:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830149.325037:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4516 00000100:00080000:0.0:1787830149.325041:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830149.325047:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830149.325058:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830149.325072:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477042560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830149.325091:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830149.325095:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830149.325096:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830149.325109:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4516 00000100:00080000:0.0:1787830149.325113:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830149.325118:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830149.325123:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830149.325124:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830149.325139:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477042688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830149.325151:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787830149.325154:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830149.325156:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830149.325160:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830149.325166:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4516 00000100:00080000:0.0:1787830149.325171:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830149.325174:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff9a7890a3c3c0 x1874671477042560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830149.325178:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830149.325188:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830149.325196:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477042560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830149.325205:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477042816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830149.325216:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830149.325218:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830149.325222:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830149.325226:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787830149.325227:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787830149.325233:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042560:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830149.325236:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4516 00000100:00080000:0.0:1787830149.325244:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830149.325248:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830149.325253:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042688:192.168.203.140@tcp:38:kworker.0 00000008:00100000:0.0:1787830149.325254:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787830149.325281:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830149.325283:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477042944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830149.325313:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff9a7890a3da40 x1874671477042688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830149.325329:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477042688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830149.325337:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830149.325339:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830149.325342:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830149.325347:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830149.325364:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830149.325374:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff9a7890a3f480 x1874671477042816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830149.325386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477042816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830149.325394:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830149.325397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830149.325400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830149.325412:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830149.325429:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830149.325443:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830149/real 1787830149] req@ffff9a7890a3de00 x1874671477042944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830149.325456:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477042944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830149.325470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830149.325474:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830149.325478:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477042944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830150.345999:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830151.370052:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830152.394111:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830153.418072:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830154.442023:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830154.443601:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830154.443609:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830154.443613:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830154.443623:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4521 00000100:00080000:2.0:1787830154.443634:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830154.443641:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830154.443672:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477043072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830154.443698:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830154.443704:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830154.443706:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830154.443709:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4521 00000100:00080000:2.0:1787830154.443712:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830154.443716:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830154.443725:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830154.443737:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477043200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830154.443746:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830154.443749:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830154.443751:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830154.443755:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4521 00000100:00080000:2.0:1787830154.443758:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830154.443761:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830154.443765:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830154.443775:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477043328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830154.443784:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830154.443787:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830154.443789:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830154.443793:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4521 00000100:00080000:2.0:1787830154.443801:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830154.443805:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830154.443810:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830154.443822:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477043456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830154.443853:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830154.443856:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830154.443858:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830154.443862:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4521 00000100:00080000:2.0:1787830154.443866:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830154.443869:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830154.443874:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830154.443895:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477043584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830154.444062:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477043072:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830154.444120:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830154.444221:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff9a78841db480 x1874671477043072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830154.444239:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477043072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830154.444257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830154.444261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830154.444265:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477043072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830154.444273:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830154.444299:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830154.444313:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff9a78841da580 x1874671477043200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830154.444325:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477043200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830154.444333:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830154.444336:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830154.444340:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830154.444346:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830154.444363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830154.444373:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff9a78841d8000 x1874671477043328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830154.444385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477043328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830154.444393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830154.444397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830154.444400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830154.444406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830154.444424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830154.444434:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff9a78841d83c0 x1874671477043456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830154.444446:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477043456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830154.444455:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830154.444458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830154.444461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830154.444467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830154.444483:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830154.444493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830154/real 1787830154] req@ffff9a78841d9e00 x1874671477043584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830154.444503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477043584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830154.444512:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830154.444514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830154.444517:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830155.465967:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830156.490280:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830157.514134:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830158.537978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830159.561995:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830159.562004:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830159.562008:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830159.562018:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4527 00000100:00080000:1.0:1787830159.562028:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830159.562035:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830159.562072:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477043712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830159.562157:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477043712:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830159.562230:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830159.562263:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830159.562268:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830159.562270:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830159.562274:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4527 00000100:00080000:1.0:1787830159.562278:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830159.562283:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830159.562293:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830159.562306:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477043840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830159.562349:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830159.564327:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830159/real 1787830159] req@ffff9a7890b1c000 x1874671477043712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830159.564357:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477043712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830159.564373:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830159.564377:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830159.564381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477043712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830159.564406:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830159.564447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830159.564462:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830159/real 1787830159] req@ffff9a7890b1f840 x1874671477043840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830159.564474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477043840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830159.564482:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830159.564485:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830159.564488:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043840:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830159.565000:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830159.565005:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830159.565008:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830159.565013:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4527 00000100:00080000:1.0:1787830159.565020:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830159.565026:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830159.565036:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830159.565058:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477043968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830159.565082:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830159.565085:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830159.565087:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830159.565091:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4527 00000100:00080000:1.0:1787830159.565095:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830159.565099:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830159.565104:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830159.565109:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830159.565114:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477044096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830159.565128:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830159.565131:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000800:00000100:0.0:1787830159.565132:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830159.565133:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830159.565137:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4527 00000100:00080000:1.0:1787830159.565140:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830159.565144:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830159/real 1787830159] req@ffff9a7890b1f0c0 x1874671477043968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830159.565144:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830159.565148:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830159.565156:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477043968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830159.565157:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477044224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830159.565166:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830159.565168:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830159.565172:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477043968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830159.565184:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830159.565202:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830159.565222:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830159/real 1787830159] req@ffff9a7890b1fc00 x1874671477044096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830159.565235:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477044096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830159.565246:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830159.565249:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830159.565252:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830159.565257:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830159.565275:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830159.565285:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830159/real 1787830159] req@ffff9a7890b1f480 x1874671477044224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830159.565296:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477044224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830159.565305:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830159.565307:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830159.565311:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830160.586048:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830161.610015:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830162.634085:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830163.658400:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830164.681990:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830164.681999:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830164.682004:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787830164.682013:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830164.682014:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4532 00000100:00080000:0.0:1787830164.682026:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830164.682032:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830164.682065:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477044352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830164.682088:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830164.682091:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830164.682093:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830164.682096:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4532 00000100:00080000:0.0:1787830164.682099:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830164.682102:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830164.682111:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830164.682121:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477044480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830164.682127:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477044352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830164.682133:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830164.682137:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830164.682139:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830164.682143:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4532 00000100:00080000:0.0:1787830164.682146:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830164.682149:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830164.682153:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830164.682165:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477044608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830164.682187:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830164.682199:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830164.682202:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787830164.682204:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830164.682208:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4532 00000100:00080000:0.0:1787830164.682212:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830164.682216:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830164.682221:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830164.682231:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477044736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830164.682239:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830164.682242:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830164.682243:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830164.682247:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4532 00000100:00080000:0.0:1787830164.682250:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830164.682256:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830164.682257:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830164/real 1787830164] req@ffff9a7890a3c000 x1874671477044352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787830164.682269:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830164.682278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477044352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830164.682282:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477044864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830164.682305:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830164.682312:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830164.682317:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477044352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830164.682340:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830164.682363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830164.682376:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830164/real 1787830164] req@ffff9a7890a3fc00 x1874671477044480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830164.682388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477044480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830164.682396:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830164.682399:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830164.682402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830164.682408:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830164.682427:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830164.682437:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830164/real 1787830164] req@ffff9a7890a3de00 x1874671477044608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830164.682448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477044608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830164.682456:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830164.682459:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830164.682463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830164.682469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830164.682488:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830164.682499:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830164/real 1787830164] req@ffff9a7890a3f480 x1874671477044736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830164.682510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477044736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830164.682524:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830164.682528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830164.682531:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830164.682536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830164.682567:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830164.682579:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830164/real 1787830164] req@ffff9a7890a3da40 x1874671477044864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830164.682590:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477044864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830164.682599:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830164.682602:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830164.682605:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477044864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830165.706034:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830166.730701:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830167.754018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830168.778062:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830169.801981:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830169.801990:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830169.801994:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830169.802003:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4537 00000100:00080000:0.0:1787830169.802014:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830169.802022:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787830169.802056:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830169.802058:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477044992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830169.802100:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830169.802103:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830169.802104:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830169.802107:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4537 00000100:00080000:0.0:1787830169.802110:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830169.802113:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830169.802120:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830169.802131:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477045120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830169.802139:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830169.802152:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830169.802154:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830169.802160:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4537 00000100:00100000:3.0:1787830169.802162:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477044992:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830169.802164:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830169.802172:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830169.802177:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830169.802193:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477045248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830169.802208:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830169.802212:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830169.802214:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830169.802220:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4537 00000100:00080000:0.0:1787830169.802231:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830169.802236:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787830169.802242:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787830169.802243:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830169.802259:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477045376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830169.802269:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830169.802272:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830169.802274:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830169.802279:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4537 00000100:00080000:0.0:1787830169.802283:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830169.802287:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830169.802291:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830169.802302:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477045504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830169.802359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830169/real 1787830169] req@ffff9a7890a3c3c0 x1874671477044992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830169.802378:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477044992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830169.802388:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830169.802392:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830169.802397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477044992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830169.802419:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830169.802455:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830169.802469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830169/real 1787830169] req@ffff9a7890a3da40 x1874671477045120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830169.802480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477045120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830169.802487:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830169.802490:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830169.802493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830169.802499:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830169.802516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830169.802526:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830169/real 1787830169] req@ffff9a7890a3f480 x1874671477045248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830169.802537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477045248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830169.802544:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830169.802547:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830169.802551:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830169.802556:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830169.802575:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830169.802585:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830169/real 1787830169] req@ffff9a7890a3de00 x1874671477045376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830169.802597:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477045376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830169.802616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830169.802619:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830169.802622:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830169.802627:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830169.802647:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830169.802657:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830169/real 1787830169] req@ffff9a7890a3fc00 x1874671477045504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830169.802669:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477045504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830169.802677:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830169.802680:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830169.802683:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830170.826403:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830171.850027:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830172.874052:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830173.898030:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830174.922015:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830174.922178:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830174.922183:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830174.922186:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830174.922194:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4542 00000100:00080000:1.0:1787830174.922203:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830174.922207:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830174.922240:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477045632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830174.922262:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830174.922265:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830174.922267:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830174.922269:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4542 00000100:00080000:1.0:1787830174.922271:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830174.922273:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830174.922281:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830174.922288:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477045760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830174.922297:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830174.922300:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830174.922302:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830174.922305:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4542 00000100:00080000:1.0:1787830174.922309:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830174.922312:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830174.922316:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830174.922326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477045888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830174.922334:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830174.922337:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830174.922338:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830174.922342:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4542 00000100:00080000:1.0:1787830174.922344:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830174.922347:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830174.922352:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830174.922361:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477046016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830174.922369:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830174.922371:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830174.922373:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830174.922376:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4542 00000100:00080000:1.0:1787830174.922378:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830174.922387:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830174.922391:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830174.922402:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477046144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830174.922474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477045632:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830174.922530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830174.922583:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830174/real 1787830174] req@ffff9a7890b1da40 x1874671477045632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830174.922603:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477045632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830174.922611:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830174.922615:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830174.922618:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477045632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830174.922625:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830174.922644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830174.922656:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830174/real 1787830174] req@ffff9a7890b1d680 x1874671477045760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830174.922680:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477045760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830174.922688:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830174.922690:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830174.922693:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830174.922698:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830174.922716:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830174.922727:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830174/real 1787830174] req@ffff9a7890b1de00 x1874671477045888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830174.922738:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477045888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830174.922750:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830174.922753:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830174.922756:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477045888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830174.922762:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830174.922779:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830174.922789:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830174/real 1787830174] req@ffff9a7890b1e580 x1874671477046016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830174.922796:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477046016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830174.922801:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830174.922803:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830174.922805:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830174.922809:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830174.922819:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830174.922824:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830174/real 1787830174] req@ffff9a7890b1e940 x1874671477046144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830174.922852:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477046144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830199 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830174.922858:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830174.922860:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830174.922862:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830175.946887:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830176.969968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830177.994394:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830179.018291:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830180.042238:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830180.042248:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830180.042252:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830180.042261:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4547 00000100:00080000:1.0:1787830180.042272:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830180.042278:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830180.042312:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477046272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830180.042345:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830180.042349:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830180.042351:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830180.042355:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4547 00000100:00080000:1.0:1787830180.042358:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830180.042361:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830180.042371:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830180.042382:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477046400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830180.042391:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830180.042393:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830180.042395:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830180.042398:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4547 00000100:00080000:1.0:1787830180.042401:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830180.042404:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830180.042408:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830180.042417:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477046528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830180.042426:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830180.042428:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830180.042430:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830180.042435:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4547 00000100:00080000:1.0:1787830180.042438:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830180.042440:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830180.042444:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830180.042452:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477046656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830180.042460:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830180.042463:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830180.042465:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830180.042468:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4547 00000100:00080000:1.0:1787830180.042471:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830180.042473:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830180.042477:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830180.042484:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477046784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830180.042674:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477046272:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830180.042729:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830180.042774:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830180/real 1787830180] req@ffff9a7890b1e940 x1874671477046272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830180.042793:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477046272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830180.042802:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830180.042806:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830180.042809:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477046272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830180.042816:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830180.042921:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830180.042934:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830180/real 1787830180] req@ffff9a7890b1e580 x1874671477046400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830180.042947:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477046400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830180.042954:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830180.042957:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830180.042959:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830180.042965:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830180.042983:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830180.042994:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830180/real 1787830180] req@ffff9a7890b1de00 x1874671477046528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830180.043004:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477046528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830180.043016:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830180.043019:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830180.043021:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830180.043027:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830180.043045:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830180.043055:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830180/real 1787830180] req@ffff9a7890b1d680 x1874671477046656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830180.043129:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477046656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830180.043139:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830180.043143:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830180.043146:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830180.043152:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830180.043172:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830180.043183:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830180/real 1787830180] req@ffff9a7890b1da40 x1874671477046784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830180.043194:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477046784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830180.043203:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830180.043205:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830180.043209:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477046784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830180.045096:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830181.066237:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830182.090964:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830183.113973:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830184.138256:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830185.162422:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830185.162432:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830185.162436:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830185.162455:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4552 00000100:00080000:1.0:1787830185.162466:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830185.162472:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830185.162512:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477046912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830185.162549:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830185.162553:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830185.162555:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830185.162559:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4552 00000100:00080000:1.0:1787830185.162562:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830185.162566:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830185.162575:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830185.162587:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477047040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830185.162597:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830185.162601:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830185.162603:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830185.162607:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4552 00000100:00080000:1.0:1787830185.162610:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830185.162614:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830185.162619:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830185.162625:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477046912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830185.162633:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477047168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830185.162644:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830185.162647:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830185.162649:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830185.162653:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4552 00000100:00080000:1.0:1787830185.162656:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830185.162659:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830185.162664:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830185.162674:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477047296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830185.162683:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830185.162686:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830185.162687:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830185.162691:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4552 00000100:00080000:1.0:1787830185.162694:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830185.162698:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830185.162702:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830185.162714:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477047424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830185.162729:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830185.162767:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830185.162788:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830185.162808:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830185.166116:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830186.186026:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830187.210869:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787830188.235730:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787830188.235770:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830188.235785:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830188.235819:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830188.235828:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830188.235879:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830188.235886:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787830188.236448:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830185/real 1787830188] req@ffff9a7890b1f0c0 x1874671477047168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830188.236472:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477047168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830188.236484:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830188.236488:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830188.236493:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830188.236499:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830185/real 1787830188] req@ffff9a7890b1f840 x1874671477047296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830188.236512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477047296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830188.236521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830188.236528:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830188.236532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830188.236537:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830185/real 1787830188] req@ffff9a7890b1c000 x1874671477047424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830188.236552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477047424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830188.236561:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830188.236564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830188.236567:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047424:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830188.236573:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830185/real 1787830188] req@ffff9a7890b1f480 x1874671477046912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830188.236584:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477046912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830188.236591:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830188.236594:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830188.236597:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477046912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830188.236602:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830185/real 1787830188] req@ffff9a7890b1fc00 x1874671477047040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830188.236612:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477047040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830188.236619:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830188.236622:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830188.236624:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047040:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787830188.237037:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830189.259302:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830190.282114:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830190.282295:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830190.282305:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830190.282310:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830190.282323:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4557 00000100:00080000:1.0:1787830190.282335:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830190.282344:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830190.282389:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477047552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830190.282425:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830190.282429:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830190.282431:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830190.282434:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4557 00000100:00080000:1.0:1787830190.282438:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830190.282442:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830190.282454:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830190.282466:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477047680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830190.282475:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830190.282478:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830190.282479:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830190.282483:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4557 00000100:00080000:1.0:1787830190.282486:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830190.282490:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830190.282494:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830190.282505:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477047808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830190.282514:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830190.282518:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830190.282520:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830190.282523:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4557 00000100:00080000:1.0:1787830190.282526:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830190.282530:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830190.282534:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830190.282544:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477047936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830190.282552:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830190.282707:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830190.282710:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830190.282715:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4557 00000100:00080000:1.0:1787830190.282719:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830190.282723:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830190.282729:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830190.282743:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477048064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830190.282865:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477047552:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830190.282963:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830190.283039:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830190/real 1787830190] req@ffff9a7890b1fc00 x1874671477047552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830190.283061:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477047552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830190.283072:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830190.283076:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830190.283082:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477047552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830190.283091:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830190.283114:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830190.283127:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830190/real 1787830190] req@ffff9a7890b1f480 x1874671477047680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830190.283138:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477047680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830190.283146:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830190.283148:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830190.283151:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830190.283156:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830190.283174:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830190.283183:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830190/real 1787830190] req@ffff9a7890b1c000 x1874671477047808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830190.283193:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477047808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830190.283206:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830190.283209:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830190.283212:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830190.283218:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830190.283234:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830190.283243:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830190/real 1787830190] req@ffff9a7890b1f840 x1874671477047936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830190.283255:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477047936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830190.283265:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830190.283268:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830190.283271:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477047936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830190.283277:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830190.283293:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830190.283302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830190/real 1787830190] req@ffff9a7890b1f0c0 x1874671477048064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830190.283312:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477048064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830190.283321:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830190.283324:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830190.283327:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830191.306152:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830192.329951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830193.354385:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830194.378136:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830195.401977:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830195.401993:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830195.402000:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830195.402004:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830195.402014:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4562 00000100:00080000:1.0:1787830195.402024:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830195.402031:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830195.402064:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477048192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830195.402134:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477048192:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830195.402181:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830195.402209:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830195.402212:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830195.402215:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830195.402228:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4562 00000100:00080000:1.0:1787830195.402232:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830195.402237:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830195.402249:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830195.402263:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477048320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830195.402390:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830195/real 1787830195] req@ffff9a7890b1da40 x1874671477048192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830195.402416:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477048192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830195.402425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830195.402428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830195.402432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477048192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830195.402454:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830195.402476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830195.402489:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830195/real 1787830195] req@ffff9a7890b1d680 x1874671477048320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830195.402500:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477048320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830195.402508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830195.402511:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830195.402514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048320:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830195.402719:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830195.402723:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830195.402725:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830195.402730:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4562 00000100:00080000:1.0:1787830195.402734:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830195.402738:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830195.402743:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830195.402756:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477048448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830195.402797:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830195.402821:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830195.402859:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830195.402863:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830195.402864:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830195.402869:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4562 00000100:00080000:1.0:1787830195.402882:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830195.402886:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830195.402891:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830195.402901:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477048576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830195.403060:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830195/real 1787830195] req@ffff9a7890b1de00 x1874671477048448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830195.403072:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477048448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830195.403079:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830195.403082:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830195.403084:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830195.403096:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830195.403115:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830195.403126:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830195/real 1787830195] req@ffff9a7890b1e580 x1874671477048576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830195.403137:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477048576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830195.403146:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830195.403150:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830195.403153:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048576:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787830195.403272:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830195.403276:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830195.403278:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830195.403282:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4562 00000100:00080000:1.0:1787830195.403286:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830195.403290:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830195.403295:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830195.403306:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477048704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830195.403345:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830195.403365:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830195.403476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830195/real 1787830195] req@ffff9a7890b1e940 x1874671477048704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830195.403493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477048704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830195.403503:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830195.403505:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830195.403509:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830196.426857:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830197.450052:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830198.473983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830199.498318:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830200.522006:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830200.522022:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830200.522028:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830200.522031:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830200.522039:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4568 00000100:00080000:2.0:1787830200.522046:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830200.522051:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830200.522090:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477048832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830200.522119:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830200.522122:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830200.522124:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830200.522128:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4568 00000100:00080000:2.0:1787830200.522137:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830200.522141:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830200.522151:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830200.522153:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477048832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830200.522165:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477048960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830200.522176:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830200.522179:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830200.522180:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830200.522184:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4568 00000100:00080000:2.0:1787830200.522188:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830200.522192:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830200.522197:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830200.522207:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477049088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830200.522215:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787830200.522220:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830200.522221:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830200.522225:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830200.522230:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4568 00000100:00080000:2.0:1787830200.522234:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830200.522238:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830200.522244:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830200.522254:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477049216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830200.522263:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830200.522267:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830200.522269:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830200.522273:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4568 00000100:00080000:2.0:1787830200.522277:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830200.522282:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830200.522283:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830200/real 1787830200] req@ffff9a78841d92c0 x1874671477048832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787830200.522289:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830200.522304:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477049344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830200.522309:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477048832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830200.522323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830200.522326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830200.522330:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477048832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830200.522353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830200.522375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830200.522386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830200/real 1787830200] req@ffff9a78841db840 x1874671477048960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830200.522396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477048960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830200.522408:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830200.522411:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830200.522414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477048960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830200.522431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830200.522450:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830200.522457:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830200/real 1787830200] req@ffff9a78841d9680 x1874671477049088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830200.522464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477049088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830200.522469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830200.522471:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830200.522473:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830200.522476:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830200.522486:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830200.522491:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830200/real 1787830200] req@ffff9a78841d8b40 x1874671477049216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830200.522497:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477049216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830200.522502:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830200.522504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830200.522506:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830200.522512:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830200.522528:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830200.522537:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830200/real 1787830200] req@ffff9a78841da940 x1874671477049344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830200.522547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477049344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830200.522556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830200.522558:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830200.522562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830201.546131:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830202.570542:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830203.594039:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830204.618006:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830205.642019:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830205.642031:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830205.642035:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830205.642045:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4573 00000100:00080000:1.0:1787830205.642057:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830205.642064:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830205.642102:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477049472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830205.642151:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477049472:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787830205.642157:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830205.642188:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830205.642192:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830205.642195:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830205.642199:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4573 00000100:00080000:1.0:1787830205.642204:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787830205.642208:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830205.642211:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830205.642223:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830205.642252:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830205/real 1787830205] req@ffff9a7890b1e940 x1874671477049472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830205.642274:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477049472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830205.642285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830205.642288:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830205.642337:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477049600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830205.642349:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830205.642352:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830205.642354:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830205.642358:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4573 00000100:00080000:1.0:1787830205.642362:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830205.642365:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830205.642480:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830205.642496:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477049728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830205.642508:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830205.642511:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830205.642514:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830205.642517:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4573 00000100:00080000:1.0:1787830205.642522:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830205.642527:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830205.642532:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477049472:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787830205.642534:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830205.642549:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477049856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830205.642555:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049600:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830205.642568:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830205.642572:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830205.642574:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830205.642580:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4573 00000800:00000100:3.0:1787830205.642581:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830205.642588:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830205.642595:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830205.642598:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830205/real 1787830205] req@ffff9a7890b1e580 x1874671477049600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787830205.642602:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830205.642610:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477049600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830205.642617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787830205.642620:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477049984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830205.642621:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830205.642626:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830205.642752:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830205.642769:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830205.642777:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830205/real 1787830205] req@ffff9a7890b1de00 x1874671477049728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830205.642785:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477049728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830205.642790:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830205.642792:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830205.642794:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830205.642803:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830205.642814:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830205.642819:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830205/real 1787830205] req@ffff9a7890b1d680 x1874671477049856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830205.642884:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477049856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830205.642891:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830205.642904:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830205.642907:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830205.642911:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830205.642925:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830205.642933:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830205/real 1787830205] req@ffff9a7890b1da40 x1874671477049984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830205.642939:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477049984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830205.642945:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830205.642947:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830205.642949:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477049984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830206.667608:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830207.690034:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830208.714986:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830209.738517:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830210.762038:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830210.762593:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830210.762601:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830210.762605:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830210.762615:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4578 00000100:00080000:3.0:1787830210.762625:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830210.762632:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830210.762668:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477050112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830210.762700:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830210.762704:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830210.762705:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830210.762709:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4578 00000100:00080000:3.0:1787830210.762713:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830210.762717:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830210.762726:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830210.762738:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477050240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830210.762747:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830210.762750:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830210.762752:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830210.762755:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4578 00000100:00080000:3.0:1787830210.762759:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830210.762763:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830210.762935:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830210.762950:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477050368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830210.762962:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830210.762967:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1787830210.762968:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477050112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830210.762970:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830210.762979:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4578 00000100:00080000:3.0:1787830210.762983:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830210.762988:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830210.762994:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830210.763006:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477050496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830210.763015:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830210.763019:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830210.763021:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830210.763024:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4578 00000100:00080000:3.0:1787830210.763027:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787830210.763032:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787830210.763034:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830210.763040:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830210.763051:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477050624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830210.763083:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830210/real 1787830210] req@ffff9a78a6776940 x1874671477050112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830210.763101:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477050112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830210.763110:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830210.763114:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830210.763119:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477050112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830210.763126:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830210.763147:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830210.763159:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830210/real 1787830210] req@ffff9a78a6776580 x1874671477050240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830210.763181:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477050240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830210.763190:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830210.763192:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830210.763196:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830210.763219:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830210.763238:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830210.763249:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830210/real 1787830210] req@ffff9a78a6774b40 x1874671477050368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830210.763485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477050368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830210.763500:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830210.763503:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830210.763506:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830210.763512:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830210.763533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830210.763544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830210/real 1787830210] req@ffff9a78a67752c0 x1874671477050496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830210.763556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477050496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830210.763565:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830210.763568:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830210.763571:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830210.763578:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830210.763595:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830210.763606:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830210/real 1787830210] req@ffff9a78a6775a40 x1874671477050624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830210.763615:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477050624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830210.763623:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830210.763626:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830210.763630:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830211.786016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830212.810069:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830213.833950:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830214.858003:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830215.882074:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830215.882083:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830215.882087:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830215.882097:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4583 00000100:00080000:1.0:1787830215.882107:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830215.882114:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830215.882155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477050752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830215.882220:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477050752:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830215.882277:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830215.882305:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830215.882308:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830215.882310:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830215.882315:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4583 00000100:00080000:1.0:1787830215.882319:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830215.882323:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830215.882332:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830215.882345:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477050880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830215.882433:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830215/real 1787830215] req@ffff9a7890b1f0c0 x1874671477050752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830215.882458:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477050752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830215.882471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830215.882474:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830215.882478:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477050752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830215.882501:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830215.882524:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830215.882536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830215/real 1787830215] req@ffff9a7890b1f840 x1874671477050880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830215.882547:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477050880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830215.882555:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830215.882558:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830215.882562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477050880:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830215.882610:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830215.882614:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830215.882617:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830215.882624:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4583 00000100:00080000:1.0:1787830215.882628:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830215.882633:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830215.882639:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830215.882653:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477051008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830215.882668:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830215.882672:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830215.882675:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830215.882679:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4583 00000100:00080000:1.0:1787830215.882683:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830215.882688:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830215.882690:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051008:192.168.203.140@tcp:38:kworker.0 00000008:00100000:1.0:1787830215.882695:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830215.882709:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477051136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830215.882719:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830215.882722:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830215.882726:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787830215.882727:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830215.882735:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4583 00000100:00080000:1.0:1787830215.882741:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830215.882745:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830215.882749:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830215.882762:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477051264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830215.882764:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830215/real 1787830215] req@ffff9a7890b1c000 x1874671477051008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830215.882795:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477051008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830215.882805:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830215.882809:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830215.882812:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830215.882861:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830215.882885:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830215.882897:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830215/real 1787830215] req@ffff9a7890b1f480 x1874671477051136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830215.882909:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477051136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830215.882919:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830215.883817:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830215.883914:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830215.883923:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830215.884036:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830215.884052:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830215/real 1787830215] req@ffff9a7890b1fc00 x1874671477051264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830215.884065:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477051264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830215.884076:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830215.884080:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830215.884083:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830215.885373:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830216.905939:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830217.930011:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830218.954214:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830219.978050:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830221.002113:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830221.002122:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830221.002126:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830221.002135:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4588 00000100:00080000:1.0:1787830221.002144:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830221.002151:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830221.002185:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477051392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830221.002213:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830221.002217:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830221.002219:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830221.002223:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4588 00000100:00080000:1.0:1787830221.002227:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830221.002232:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830221.002240:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830221.002264:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477051520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830221.002274:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830221.002277:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830221.002279:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830221.002283:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4588 00000100:00080000:1.0:1787830221.002286:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830221.002290:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830221.002294:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830221.002305:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477051648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830221.002314:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830221.002317:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830221.002319:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830221.002322:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4588 00000100:00080000:1.0:1787830221.002325:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830221.002328:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830221.002332:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830221.002341:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477051776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830221.002349:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830221.002352:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830221.002353:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830221.002356:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4588 00000100:00080000:1.0:1787830221.002359:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830221.002363:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830221.002367:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830221.002379:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477051904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830221.002457:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477051392:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830221.002511:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830221.002554:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830221/real 1787830221] req@ffff9a7890b1fc00 x1874671477051392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830221.002569:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477051392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830221.002579:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830221.002582:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830221.002586:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477051392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830221.002594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830221.002616:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830221.002627:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830221/real 1787830221] req@ffff9a7890b1f480 x1874671477051520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830221.002637:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477051520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830221.002643:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830221.002646:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830221.002649:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830221.002654:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830221.002672:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830221.002683:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830221/real 1787830221] req@ffff9a7890b1c000 x1874671477051648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830221.002695:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477051648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830221.002702:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830221.002705:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830221.002708:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830221.002713:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830221.002729:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830221.002739:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830221/real 1787830221] req@ffff9a7890b1f840 x1874671477051776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830221.002751:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477051776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830221.002760:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830221.002762:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830221.002765:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830221.002771:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830221.002804:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830221.002821:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830221/real 1787830221] req@ffff9a7890b1f0c0 x1874671477051904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830221.002857:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477051904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830221.002871:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830221.002874:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830221.002878:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477051904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830221.002907:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830222.027596:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830223.050230:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830224.075403:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830225.098033:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830226.121958:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830226.121968:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830226.121972:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830226.121982:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4593 00000100:00080000:1.0:1787830226.121993:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830226.122000:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830226.122036:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477052032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830226.122064:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830226.122067:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830226.122069:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830226.122073:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4593 00000100:00080000:1.0:1787830226.122076:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830226.122080:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830226.122089:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830226.122102:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477052160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830226.122113:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787830226.122117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477052032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830226.122117:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830226.122121:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830226.122129:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4593 00000100:00080000:1.0:1787830226.122133:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830226.122137:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830226.122141:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830226.122156:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477052288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830226.122166:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830226.122169:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830226.122172:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830226.122176:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4593 00000100:00080000:1.0:1787830226.122179:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830226.122185:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787830226.122187:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787830226.122193:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830226.122207:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477052416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830226.122217:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830226.122220:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830226.122221:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830226.122225:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4593 00000100:00080000:1.0:1787830226.122228:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830226.122232:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830226.122237:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830226.122249:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477052544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830226.122262:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830226/real 1787830226] req@ffff9a7890b1da40 x1874671477052032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830226.122284:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477052032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830226.122295:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830226.122299:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830226.122303:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477052032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830226.122326:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830226.122348:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830226.122366:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830226/real 1787830226] req@ffff9a7890b1d680 x1874671477052160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830226.122378:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477052160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830226.122392:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830226.122395:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830226.122398:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830226.122404:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830226.122422:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830226.122432:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830226/real 1787830226] req@ffff9a7890b1de00 x1874671477052288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830226.122443:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477052288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830226.122462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830226.122466:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830226.122469:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830226.122475:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830226.122494:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830226.122504:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830226/real 1787830226] req@ffff9a7890b1e580 x1874671477052416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830226.122514:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477052416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830226.122523:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830226.122526:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830226.122530:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830226.122537:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830226.122554:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830226.122564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830226/real 1787830226] req@ffff9a7890b1e940 x1874671477052544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830226.122574:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477052544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830226.122583:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830226.122585:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830226.122589:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830226.123353:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830227.146126:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830228.170050:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830229.193980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830230.219255:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830231.241942:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830231.241951:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830231.241955:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830231.241965:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4598 00000100:00080000:1.0:1787830231.241975:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830231.241990:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830231.242027:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477052672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830231.242057:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830231.242060:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830231.242062:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830231.242066:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4598 00000100:00080000:1.0:1787830231.242070:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830231.242074:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830231.242082:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830231.242094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477052800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830231.242103:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830231.242106:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830231.242108:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830231.242111:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4598 00000100:00080000:1.0:1787830231.242113:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830231.242117:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830231.242121:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830231.242131:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477052928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830231.242139:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830231.242143:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830231.242144:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830231.242147:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4598 00000100:00080000:1.0:1787830231.242150:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830231.242153:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830231.242158:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830231.242167:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477053056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830231.242175:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830231.242178:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830231.242179:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830231.242188:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4598 00000100:00080000:1.0:1787830231.242191:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830231.242194:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830231.242198:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830231.242208:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477053184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830231.242396:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830231.242484:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477052672:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830231.242551:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830231.242592:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830231/real 1787830231] req@ffff9a7890b1e940 x1874671477052672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830231.242607:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477052672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830231.242616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830231.242619:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830231.242623:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477052672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830231.242638:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830231.242657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830231.242668:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830231/real 1787830231] req@ffff9a7890b1e580 x1874671477052800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830231.242679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477052800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830231.242686:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830231.242689:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830231.242692:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830231.242698:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830231.242712:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830231.242734:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830231/real 1787830231] req@ffff9a7890b1de00 x1874671477052928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830231.242745:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477052928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830231.242753:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830231.242755:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830231.242758:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477052928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830231.242764:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830231.242780:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830231.242788:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830231/real 1787830231] req@ffff9a7890b1d680 x1874671477053056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830231.242799:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477053056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830231.242808:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830231.242810:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830231.242813:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830231.242818:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830231.242866:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830231.242877:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830231/real 1787830231] req@ffff9a7890b1da40 x1874671477053184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830231.242888:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477053184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830231.242896:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830231.242899:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830231.242902:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830232.265980:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830233.290066:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830234.314157:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830235.338865:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830236.362276:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830236.362287:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830236.362291:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830236.362301:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4603 00000100:00080000:3.0:1787830236.362311:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830236.362318:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830236.362355:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671477053312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830236.362383:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830236.362387:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830236.362389:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830236.362393:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4603 00000100:00080000:3.0:1787830236.362396:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830236.362401:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830236.362410:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830236.362422:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671477053440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830236.362431:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830236.362435:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830236.362437:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830236.362441:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4603 00000100:00080000:3.0:1787830236.362444:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830236.362447:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830236.362452:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830236.362460:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477053312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830236.362466:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477053568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830236.362483:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830236.362486:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830236.362488:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830236.362493:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4603 00000100:00080000:3.0:1787830236.362511:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830236.362516:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830236.362520:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787830236.362521:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830236.362534:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477053696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830236.362544:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830236.362547:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830236.362549:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830236.362553:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4603 00000100:00080000:3.0:1787830236.362556:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830236.362559:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830236.362564:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830236.362575:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477053824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830236.362585:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830236/real 1787830236] req@ffff9a78a6774000 x1874671477053312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830236.362605:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671477053312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830236.362616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830236.362620:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830236.362624:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477053312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830236.362632:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830236.362668:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830236.362693:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830236/real 1787830236] req@ffff9a78a6777480 x1874671477053440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830236.362708:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671477053440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787830236.362715:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830236.362718:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830236.362723:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830236.362727:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830236.362763:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830236.362784:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830236.362809:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830236/real 1787830236] req@ffff9a78a6775a40 x1874671477053568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830236.362821:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477053568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830236.362852:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830236.362857:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830236.362860:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830236.362866:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830236.362886:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830236.362898:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830236/real 1787830236] req@ffff9a78a67752c0 x1874671477053696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830236.362909:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477053696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830236.362918:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830236.362921:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830236.362924:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830236.362930:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830236.362961:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830236.362973:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830236/real 1787830236] req@ffff9a78a6774b40 x1874671477053824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830236.362986:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477053824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830236.362995:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830236.362998:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830236.363001:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477053824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830237.386058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830238.410542:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830239.434956:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830240.458585:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830241.482036:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787830241.482049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830241.482057:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830241.482062:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830241.482072:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4608 00000100:00080000:0.0:1787830241.482082:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830241.482098:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830241.482134:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477053952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830241.482161:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830241.482165:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830241.482166:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830241.482170:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4608 00000100:00080000:0.0:1787830241.482174:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830241.482178:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830241.482187:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830241.482200:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477054080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830241.482210:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830241.482214:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830241.482216:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830241.482220:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4608 00000100:00080000:0.0:1787830241.482223:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830241.482229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477053952:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830241.482230:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830241.482238:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830241.482252:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477054208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830241.482261:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830241.482264:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830241.482266:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830241.482280:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4608 00000100:00080000:0.0:1787830241.482294:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787830241.482299:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830241.482301:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830241.482308:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830241.482321:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477054336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830241.482331:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830241.482334:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830241.482336:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830241.482341:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4608 00000100:00080000:0.0:1787830241.482344:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830241.482348:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830241.482353:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830241.482365:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477054464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830241.482372:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830241/real 1787830241] req@ffff9a7890a3c000 x1874671477053952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830241.482392:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477053952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830241.482402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830241.482406:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830241.482409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477053952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830241.482416:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830241.482436:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830241.482454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830241/real 1787830241] req@ffff9a7890a3fc00 x1874671477054080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830241.482465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477054080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830241.482473:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830241.482476:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830241.482479:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830241.482500:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830241.482517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830241.482527:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830241/real 1787830241] req@ffff9a7890a3de00 x1874671477054208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830241.482537:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477054208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830241.482549:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830241.482552:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830241.482555:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830241.482560:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830241.482575:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830241.482583:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830241/real 1787830241] req@ffff9a7890a3f480 x1874671477054336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830241.482593:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477054336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830241.482601:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830241.482604:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830241.482606:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830241.482612:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830241.482631:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830241.482640:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830241/real 1787830241] req@ffff9a7890a3da40 x1874671477054464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830241.482662:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477054464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830241.482671:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830241.482674:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830241.482677:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830242.506047:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830243.530091:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830244.554634:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830245.579044:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830246.602020:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830246.602049:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830246.602056:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830246.602060:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830246.602070:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4614 00000100:00080000:2.0:1787830246.602079:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830246.602085:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830246.602119:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477054592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830246.602146:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830246.602149:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830246.602151:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830246.602155:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4614 00000100:00080000:2.0:1787830246.602158:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830246.602162:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830246.602171:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830246.602184:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477054720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830246.602192:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830246.602195:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830246.602196:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830246.602200:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4614 00000100:00080000:2.0:1787830246.602203:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830246.602206:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830246.602210:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830246.602221:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477054848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830246.602230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830246.602233:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830246.602234:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830246.602238:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4614 00000100:00080000:2.0:1787830246.602241:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830246.602243:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830246.602248:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830246.602257:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477054976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830246.602267:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830246.602270:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830246.602272:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830246.602276:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4614 00000100:00080000:2.0:1787830246.602280:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830246.602284:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830246.602289:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830246.602300:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477054592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830246.602301:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477055104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830246.602362:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830246.602410:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830246/real 1787830246] req@ffff9a78841da940 x1874671477054592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830246.602425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477054592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830246.602434:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830246.602438:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830246.602441:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477054592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830246.602447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830246.602464:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830246.602474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830246/real 1787830246] req@ffff9a78841d8b40 x1874671477054720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830246.602483:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477054720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830246.602490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830246.602493:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830246.602496:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830246.602500:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830246.602514:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830246.602521:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830246/real 1787830246] req@ffff9a78841d9680 x1874671477054848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830246.602530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477054848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830246.602537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830246.602539:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830246.602542:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830246.602546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830246.602557:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830246.602565:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830246/real 1787830246] req@ffff9a78841db840 x1874671477054976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830246.602574:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477054976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830246.602582:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830246.602584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830246.602587:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477054976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830246.602618:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830246.602632:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830246.602640:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830246/real 1787830246] req@ffff9a78841d92c0 x1874671477055104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830246.602655:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477055104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830246.602663:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830246.602666:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830246.602668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830247.625973:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830248.650177:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830249.674192:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830250.697974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830251.721955:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830251.721964:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830251.721968:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830251.721978:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4619 00000100:00080000:2.0:1787830251.721989:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830251.721997:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830251.722032:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477055232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830251.722061:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830251.722066:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830251.722069:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830251.722074:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4619 00000100:00080000:2.0:1787830251.722078:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830251.722093:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830251.722103:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830251.722119:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477055360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830251.722134:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830251.722138:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830251.722141:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830251.722158:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4619 00000100:00100000:0.0:1787830251.722160:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477055232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830251.722163:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830251.722174:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830251.722181:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830251.722194:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477055488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830251.722204:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830251.722208:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830251.722210:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830251.722214:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4619 00000100:00080000:2.0:1787830251.722218:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830251.722223:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830251.722227:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830251.722238:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477055616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830251.722248:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830251.722251:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830251.722254:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830251.722260:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4619 00000100:00080000:2.0:1787830251.722276:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830251.722281:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830251.722287:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830251.722301:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477055744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830251.722312:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830251.722361:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830251.722389:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830251.722412:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830251.723082:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830252.747697:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830253.770118:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830254.794166:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787830254.794307:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787830254.794358:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787830254.794377:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787830254.795639:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830251/real 1787830254] req@ffff9a78841d9e00 x1874671477055232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830276 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830254.795677:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477055232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830276 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830254.795692:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830254.795698:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830254.795703:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477055232:192.168.203.140@tcp:250:kworker.0 00000800:00000100:2.0:1787830254.795826:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830254.795874:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830254.795884:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830254.795892:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787830254.796012:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830251/real 1787830254] req@ffff9a78841d8000 x1874671477055488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830254.796024:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477055488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830254.796030:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830254.796039:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830254.796043:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830254.796048:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830251/real 1787830254] req@ffff9a78841da580 x1874671477055616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830254.796057:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477055616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830254.796064:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830254.796065:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830254.796067:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830254.796071:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830251/real 1787830254] req@ffff9a78841db480 x1874671477055744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830254.796079:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477055744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830254.796085:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830254.796087:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830254.796089:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055744:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830254.796092:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830251/real 1787830254] req@ffff9a78841d83c0 x1874671477055360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830254.796098:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477055360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830254.796102:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830254.796104:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830254.796106:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477055360:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787830255.818477:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830256.841994:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830256.842003:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830256.842008:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830256.842018:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4624 00000100:00080000:0.0:1787830256.842029:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830256.842036:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787830256.842036:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830256.842072:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477055872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830256.842102:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830256.842105:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830256.842107:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830256.842111:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4624 00000100:00080000:0.0:1787830256.842115:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830256.842119:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830256.842128:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830256.842142:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477056000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830256.842150:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477055872:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830256.842156:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830256.842161:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830256.842163:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830256.842168:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4624 00000100:00080000:0.0:1787830256.842172:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830256.842175:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830256.842181:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830256.842193:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477056128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830256.842202:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830256.842205:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830256.842208:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787830256.842212:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830256.842216:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4624 00000100:00080000:0.0:1787830256.842221:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830256.842225:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830256.842231:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830256.842253:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477056256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830256.842261:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830256/real 1787830256] req@ffff9a7890a3c3c0 x1874671477055872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830256.842266:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830256.842279:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830256.842284:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787830256.842284:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477055872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830256.842291:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4624 00000100:00080000:0.0:1787830256.842299:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830256.842303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830256.842305:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787830256.842311:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000008:00100000:0.0:1787830256.842313:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830256.842316:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477055872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830256.842330:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477056384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830256.842357:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830256.842379:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830256.842391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830256/real 1787830256] req@ffff9a7890a3da40 x1874671477056000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830256.842415:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477056000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830256.842423:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830256.842426:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830256.842429:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830256.842435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830256.842459:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830256.842469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830256/real 1787830256] req@ffff9a7890a3f480 x1874671477056128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830256.842485:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477056128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830256.842493:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830256.842496:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830256.842499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830256.842504:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830256.842523:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830256.842534:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830256/real 1787830256] req@ffff9a7890a3de00 x1874671477056256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830256.842555:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477056256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830256.842570:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830256.842573:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830256.842576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830256.842581:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830256.842599:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830256.842615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830256/real 1787830256] req@ffff9a7890a3fc00 x1874671477056384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830256.842626:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477056384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830256.842634:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830256.842637:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830256.842639:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830257.867591:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830258.890171:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830259.913991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830260.938731:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830261.962028:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830261.962131:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830261.962136:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830261.962145:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4629 00000100:00080000:1.0:1787830261.962155:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830261.962161:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830261.962197:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477056512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830261.962218:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830261.962221:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830261.962223:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830261.962227:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4629 00000100:00080000:1.0:1787830261.962230:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830261.962234:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830261.962242:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830261.962253:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477056640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830261.962261:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830261.962264:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830261.962266:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830261.962270:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4629 00000100:00080000:1.0:1787830261.962273:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830261.962276:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830261.962280:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830261.962289:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477056768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830261.962297:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830261.962300:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830261.962301:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830261.962305:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4629 00000100:00080000:1.0:1787830261.962308:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830261.962311:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830261.962316:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830261.962325:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477056896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830261.962333:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830261.962336:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830261.962338:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830261.962342:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4629 00000100:00080000:1.0:1787830261.962346:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830261.962349:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830261.962353:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830261.962444:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477057024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830261.962526:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477056512:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830261.962580:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830261.962690:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830261/real 1787830261] req@ffff9a7890b1f0c0 x1874671477056512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830261.962707:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477056512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830261.962717:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830261.962720:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830261.962724:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477056512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830261.962731:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830261.962751:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830261.962762:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830261/real 1787830261] req@ffff9a7890b1f840 x1874671477056640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830261.962773:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477056640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830261.962781:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830261.962784:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830261.962786:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830261.962792:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830261.962807:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830261.962816:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830261/real 1787830261] req@ffff9a7890b1c000 x1874671477056768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830261.962826:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477056768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830261.962870:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830261.962873:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830261.962876:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830261.962881:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830261.962898:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830261.962908:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830261/real 1787830261] req@ffff9a7890b1f480 x1874671477056896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830261.962918:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477056896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830261.962927:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830261.962929:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830261.962932:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477056896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830261.962936:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830261.962953:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830261.962964:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830261/real 1787830261] req@ffff9a7890b1fc00 x1874671477057024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830261.962973:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477057024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830286 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830261.962981:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830261.962984:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830261.962986:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830261.963127:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830262.986011:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830264.010168:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830265.034094:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830266.058018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830267.081972:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830267.081984:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830267.081989:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830267.082008:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4634 00000100:00080000:1.0:1787830267.082021:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830267.082030:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830267.082098:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477057152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830267.082126:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830267.082130:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830267.082132:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830267.082136:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4634 00000100:00080000:1.0:1787830267.082139:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830267.082143:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830267.082154:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830267.082167:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477057280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830267.082177:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830267.082180:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830267.082183:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830267.082187:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4634 00000100:00080000:1.0:1787830267.082190:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830267.082194:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830267.082198:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830267.082221:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477057408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830267.082228:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830267.082232:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830267.082236:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830267.082239:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830267.082243:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4634 00000100:00080000:1.0:1787830267.082247:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830267.082251:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830267.082257:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830267.082267:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477057536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830267.082277:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830267.082279:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830267.082281:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830267.082294:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4634 00000100:00080000:1.0:1787830267.082298:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830267.082302:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830267.082306:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830267.082318:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477057664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830267.082415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477057152:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830267.082511:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830267.082579:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830267/real 1787830267] req@ffff9a7890b1fc00 x1874671477057152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830267.082606:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477057152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830267.082617:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830267.082622:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830267.082626:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477057152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830267.082634:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830267.082656:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830267.082668:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830267/real 1787830267] req@ffff9a7890b1f480 x1874671477057280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830267.082679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477057280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830267.082687:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830267.082691:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830267.082694:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830267.082699:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830267.082727:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830267.082738:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830267/real 1787830267] req@ffff9a7890b1c000 x1874671477057408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830267.082749:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477057408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830267.082762:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830267.082766:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830267.082770:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830267.082775:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830267.082794:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830267.082805:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830267/real 1787830267] req@ffff9a7890b1f840 x1874671477057536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830267.082816:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477057536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830267.082826:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830267.082851:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830267.082854:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830267.082860:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830267.082880:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830267.082890:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830267/real 1787830267] req@ffff9a7890b1f0c0 x1874671477057664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830267.082901:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477057664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830267.082911:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830267.082914:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830267.082917:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830268.106123:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830269.129968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830270.154862:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830271.178501:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830272.202021:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830272.202040:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830272.202049:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830272.202067:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4639 00000100:00080000:1.0:1787830272.202076:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830272.202083:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830272.202130:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477057792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830272.202154:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830272.202212:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830272.202214:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830272.202218:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4639 00000100:00080000:1.0:1787830272.202221:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830272.202226:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830272.202235:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830272.202247:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477057920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830272.202256:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830272.202259:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830272.202261:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830272.202283:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4639 00000100:00080000:1.0:1787830272.202287:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830272.202291:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830272.202295:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830272.202306:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477058048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830272.202325:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830272.202329:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830272.202331:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830272.202334:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4639 00000100:00080000:1.0:1787830272.202337:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830272.202341:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830272.202346:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830272.202358:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477058176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830272.202367:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830272.202369:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830272.202371:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830272.202374:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4639 00000100:00080000:1.0:1787830272.202377:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830272.202380:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830272.202383:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830272.202390:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477058304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830272.203656:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477057792:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830272.203711:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830272.203777:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830272/real 1787830272] req@ffff9a7890b1da40 x1874671477057792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830272.203794:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477057792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830272.203804:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830272.203809:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830272.203812:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477057792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830272.203820:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830272.203870:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830272.203882:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830272/real 1787830272] req@ffff9a7890b1d680 x1874671477057920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830272.203893:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477057920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830272.203901:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830272.203903:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830272.203906:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477057920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830272.203912:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830272.203931:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830272.203942:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830272/real 1787830272] req@ffff9a7890b1de00 x1874671477058048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830272.203954:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477058048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830272.203962:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830272.203966:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830272.203969:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830272.203975:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830272.203993:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830272.204002:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830272/real 1787830272] req@ffff9a7890b1e580 x1874671477058176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830272.204013:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477058176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830272.204023:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830272.204026:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830272.204029:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830272.204035:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830272.204050:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830272.204060:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830272/real 1787830272] req@ffff9a7890b1e940 x1874671477058304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830272.204078:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477058304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830272.204087:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830272.204090:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830272.204093:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830272.204205:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830273.226016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830274.250113:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830275.273965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830276.298048:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830277.321979:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830277.321989:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830277.321994:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830277.322004:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4644 00010000:00010000:2.0:1787830277.322008:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830277.322016:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830277.322024:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830277.322064:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477058432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830277.322092:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830277.322097:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830277.322099:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830277.322102:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4644 00000100:00080000:1.0:1787830277.322106:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830277.322110:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830277.322120:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830277.322133:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477058560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830277.322142:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830277.322145:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830277.322147:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830277.322151:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4644 00000100:00080000:1.0:1787830277.322154:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830277.322158:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830277.322162:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830277.322173:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477058688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830277.322180:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830277.322183:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830277.322185:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830277.322188:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4644 00000100:00080000:1.0:1787830277.322191:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830277.322194:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830277.322199:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830277.322207:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477058816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830277.322214:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830277.322217:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830277.322219:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830277.322222:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4644 00000100:00080000:1.0:1787830277.322224:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830277.322227:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830277.322231:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830277.322239:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477058944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830277.322288:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477058432:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830277.322376:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830277.322441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830277/real 1787830277] req@ffff9a7890b1e940 x1874671477058432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830277.322459:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477058432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830277.322469:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830277.322473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830277.322477:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477058432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830277.322485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830277.322505:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830277.322517:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830277/real 1787830277] req@ffff9a7890b1e580 x1874671477058560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830277.322527:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477058560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830277.322534:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830277.322537:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830277.322540:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830277.322545:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830277.322562:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830277.322571:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830277/real 1787830277] req@ffff9a7890b1de00 x1874671477058688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830277.322582:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477058688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830277.322591:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830277.322593:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830277.322596:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830277.322602:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830277.322635:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830277.322646:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830277/real 1787830277] req@ffff9a7890b1d680 x1874671477058816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830277.322657:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477058816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830277.322666:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830277.322669:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830277.322672:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830277.322678:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830277.322696:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830277.322706:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830277/real 1787830277] req@ffff9a7890b1da40 x1874671477058944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830277.322715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477058944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830277.322724:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830277.322727:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830277.322730:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477058944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830278.345974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830279.370106:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830280.394228:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830281.418225:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830282.441959:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830282.441976:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830282.441983:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830282.441987:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830282.441996:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4649 00000100:00080000:1.0:1787830282.442006:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830282.442012:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830282.442049:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477059072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830282.442076:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830282.442079:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830282.442081:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830282.442085:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4649 00000100:00080000:1.0:1787830282.442088:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830282.442092:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830282.442110:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830282.442125:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477059200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830282.442134:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830282.442136:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830282.442138:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830282.442140:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4649 00000100:00080000:1.0:1787830282.442143:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830282.442147:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830282.442151:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830282.442161:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477059328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830282.442170:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830282.442174:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830282.442175:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830282.442179:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4649 00000100:00080000:1.0:1787830282.442183:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830282.442186:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830282.442190:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830282.442200:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477059456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830282.442399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477059072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830282.442400:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830282.442407:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830282.442410:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830282.442415:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4649 00000100:00080000:1.0:1787830282.442419:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830282.442423:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830282.442428:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830282.442438:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477059584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830282.442466:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830282.442513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830282/real 1787830282] req@ffff9a7890b1f0c0 x1874671477059072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830282.442530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477059072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830282.442540:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830282.442544:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830282.442547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477059072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830282.442554:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830282.442572:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830282.442583:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830282/real 1787830282] req@ffff9a7890b1f840 x1874671477059200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830282.442594:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477059200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830282.442601:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830282.442605:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830282.442607:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830282.442613:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830282.442630:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830282.442642:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830282/real 1787830282] req@ffff9a7890b1c000 x1874671477059328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830282.442652:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477059328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830282.442660:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830282.442662:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830282.442665:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830282.442709:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830282.442730:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830282.442742:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830282/real 1787830282] req@ffff9a7890b1f480 x1874671477059456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830282.442759:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477059456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830282.442769:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830282.442772:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830282.442775:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830282.442780:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830282.442799:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830282.442810:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830282/real 1787830282] req@ffff9a7890b1fc00 x1874671477059584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830282.442821:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477059584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830282.442853:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830282.442857:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830282.442860:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830283.466116:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830284.490507:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830285.514114:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830286.538687:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830287.561966:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830287.561976:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830287.561981:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830287.561992:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4655 00000100:00080000:1.0:1787830287.562004:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830287.562012:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787830287.562016:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830287.562059:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477059712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830287.562084:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830287.562086:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830287.562088:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830287.562090:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4655 00000100:00080000:1.0:1787830287.562092:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830287.562095:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830287.562104:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830287.562111:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477059840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830287.562117:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830287.562119:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830287.562120:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830287.562123:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4655 00000100:00080000:1.0:1787830287.562124:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830287.562126:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830287.562128:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830287.562134:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477059968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830287.562139:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830287.562142:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830287.562143:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830287.562145:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4655 00000100:00080000:1.0:1787830287.562147:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830287.562149:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830287.562152:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830287.562157:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477060096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830287.562161:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830287.562163:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830287.562164:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830287.562166:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4655 00000100:00080000:1.0:1787830287.562167:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830287.562169:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830287.562171:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830287.562176:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477060224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830287.562237:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477059712:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830287.562312:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830287.562378:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830287/real 1787830287] req@ffff9a7890b1fc00 x1874671477059712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830287.562397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477059712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830287.562409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830287.562414:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830287.562418:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477059712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830287.562425:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830287.562448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830287.562459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830287/real 1787830287] req@ffff9a7890b1f480 x1874671477059840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830287.562471:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477059840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830287.562479:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830287.562482:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830287.562485:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830287.562491:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830287.562509:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830287.562519:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830287/real 1787830287] req@ffff9a7890b1c000 x1874671477059968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830287.562530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477059968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830287.562537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830287.562541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830287.562544:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477059968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830287.562550:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830287.562569:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830287.562580:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830287/real 1787830287] req@ffff9a7890b1f840 x1874671477060096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830287.562591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477060096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830287.562600:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830287.562604:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830287.562607:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830287.562613:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830287.562631:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830287.562642:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830287/real 1787830287] req@ffff9a7890b1f0c0 x1874671477060224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830287.562653:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477060224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830287.562661:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830287.562664:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830287.562667:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830288.591596:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830289.610185:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830290.633983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830291.659582:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830292.682006:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830292.682015:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830292.682019:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830292.682030:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4660 00010000:00010000:2.0:1787830292.682035:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830292.682040:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830292.682047:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830292.682084:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477060352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830292.682111:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830292.682114:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830292.682116:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830292.682119:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4660 00000100:00080000:3.0:1787830292.682123:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830292.682127:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830292.682137:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830292.682146:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477060352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830292.682153:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477060480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830292.682165:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830292.682168:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830292.682170:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830292.682175:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4660 00000100:00080000:3.0:1787830292.682192:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830292.682197:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787830292.682201:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787830292.682211:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830292.682225:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671477060608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830292.682234:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830292.682238:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1787830292.682240:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830292/real 1787830292] req@ffff9a78a6776580 x1874671477060352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830292.682241:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830292.682250:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4660 00000100:00080000:3.0:1787830292.682257:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830292.682261:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830292.682263:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477060352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1787830292.682267:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:0.0:1787830292.682280:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1787830292.682284:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671477060736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830292.682304:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830292.682310:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477060352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830292.682312:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830292.682318:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830292.682320:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830292.682326:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4660 00000100:00080000:3.0:1787830292.682330:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830292.682334:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830292.682338:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060480:192.168.203.140@tcp:38:kworker.0 00000008:00100000:3.0:1787830292.682339:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830292.682354:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477060864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830292.682363:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830292.682377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830292/real 1787830292] req@ffff9a78a6776940 x1874671477060480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830292.682385:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477060480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830292.682391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830292.682393:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830292.682395:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830292.682399:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830292.682409:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830292.682414:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830292/real 1787830292] req@ffff9a78a67770c0 x1874671477060608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830292.682421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671477060608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830292.682425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830292.682427:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830292.682428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830292.682431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830292.682440:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830292.682445:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830292/real 1787830292] req@ffff9a78a6775680 x1874671477060736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830292.682455:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671477060736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830292.682461:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830292.682462:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830292.682464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830292.682472:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830292.682481:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830292.682486:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830292/real 1787830292] req@ffff9a78a6774780 x1874671477060864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830292.682492:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477060864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830292.682497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830292.682499:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830292.682500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477060864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830293.706973:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830294.730008:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830295.754618:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830296.778029:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830297.801974:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830297.801985:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830297.801990:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830297.801999:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4665 00000100:00080000:3.0:1787830297.802014:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830297.802021:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830297.802055:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477060992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830297.802083:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830297.802087:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830297.802089:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830297.802092:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4665 00000100:00080000:3.0:1787830297.802095:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830297.802105:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830297.802118:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830297.802130:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477061120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830297.802139:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830297.802142:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830297.802144:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830297.802147:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4665 00000100:00080000:3.0:1787830297.802151:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830297.802154:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830297.802158:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830297.802171:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671477061248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830297.802185:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830297.802188:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830297.802190:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830297.802194:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4665 00000100:00080000:3.0:1787830297.802197:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830297.802200:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830297.802205:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830297.802215:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671477061376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830297.802224:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830297.802227:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830297.802229:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830297.802232:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4665 00000100:00080000:3.0:1787830297.802236:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830297.802240:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830297.802244:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830297.802266:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671477061504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830297.802441:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477060992:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830297.802496:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830297.802548:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830297/real 1787830297] req@ffff9a78a67761c0 x1874671477060992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830297.802564:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477060992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830297.802579:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830297.802582:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830297.802586:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477060992:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787830297.802724:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830297.803292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830297.803355:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830297.803379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830297/real 1787830297] req@ffff9a78a6776d00 x1874671477061120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830297.803391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477061120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830297.803407:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830297.803412:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830297.803415:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830297.803421:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830297.803441:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830297.803452:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830297/real 1787830297] req@ffff9a78a6775e00 x1874671477061248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830297.803465:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671477061248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830297.803470:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830297.803473:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830297.803477:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830297.803481:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830297.803498:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830297.803508:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830297/real 1787830297] req@ffff9a78a6777c00 x1874671477061376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830297.803519:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671477061376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830297.803529:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830297.803532:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830297.803536:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830297.803570:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830297.803585:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830297.803593:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830297/real 1787830297] req@ffff9a78a6777840 x1874671477061504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830297.803604:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671477061504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830297.803614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830297.803617:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830297.803620:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830298.826135:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830299.849954:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830300.877143:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830301.898449:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830302.921959:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830302.921968:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830302.921972:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830302.921983:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4670 00000100:00080000:3.0:1787830302.921994:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830302.922000:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830302.922036:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671477061632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830302.922068:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830302.922072:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830302.922073:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830302.922077:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4670 00000100:00080000:3.0:1787830302.922080:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830302.922085:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830302.922094:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830302.922105:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671477061760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830302.922113:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830302.922116:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830302.922118:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830302.922121:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4670 00000100:00080000:3.0:1787830302.922124:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830302.922126:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830302.922131:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830302.922142:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671477061888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830302.922150:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830302.922153:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830302.922155:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830302.922158:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4670 00000100:00080000:3.0:1787830302.922160:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830302.922164:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830302.922168:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830302.922190:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671477062016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830302.922198:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830302.922201:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830302.922203:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830302.922206:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4670 00000100:00080000:3.0:1787830302.922209:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830302.922213:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830302.922230:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830302.922241:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671477062144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787830302.922315:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830302.922321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477061632:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830302.922468:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830302.922527:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830302/real 1787830302] req@ffff9a78a6774f00 x1874671477061632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830302.922546:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671477061632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830302.922556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830302.922560:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830302.922563:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477061632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830302.922568:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830302.922586:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830302.922594:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830302/real 1787830302] req@ffff9a78a67743c0 x1874671477061760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830302.922602:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671477061760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830302.922607:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830302.922609:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830302.922611:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830302.922614:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830302.922685:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830302.922696:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830302/real 1787830302] req@ffff9a78a6777840 x1874671477061888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830302.922709:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671477061888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830302.922723:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830302.922727:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830302.922729:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477061888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830302.922735:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830302.922754:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830302.922764:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830302/real 1787830302] req@ffff9a78a6777c00 x1874671477062016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830302.922774:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671477062016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830302.922783:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830302.922786:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830302.922789:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830302.922795:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830302.922808:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830302.922815:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830302/real 1787830302] req@ffff9a78a6775e00 x1874671477062144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830302.922822:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671477062144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830327 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830302.922827:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830302.922865:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830302.922869:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830303.946069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830304.972746:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830305.993966:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830307.018043:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830308.042785:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830308.042794:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830308.042798:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830308.042807:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4675 00000100:00080000:0.0:1787830308.042817:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830308.042823:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:2.0:1787830308.043099:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830308.043393:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477062272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830308.043469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477062272:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830308.043528:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830308.043559:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830308.043564:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830308.043566:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830308.043575:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4675 00000100:00080000:0.0:1787830308.043580:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830308.043585:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830308.043596:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830308.043610:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477062400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830308.043767:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830308/real 1787830308] req@ffff9a7890a3c000 x1874671477062272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830308.043791:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477062272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830308.043801:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830308.043805:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830308.043809:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477062272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830308.043866:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830308.043907:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830308.043921:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830308/real 1787830308] req@ffff9a7890a3fc00 x1874671477062400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830308.043933:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477062400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830308.043951:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830308.043955:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830308.043958:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062400:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830308.045347:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830308.045355:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830308.045358:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830308.045365:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4675 00000100:00080000:0.0:1787830308.045372:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830308.045378:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830308.045387:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830308.045408:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477062528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830308.045428:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830308.045432:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830308.045434:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830308.045438:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4675 00000100:00080000:0.0:1787830308.045441:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830308.045445:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830308.045449:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830308.045458:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477062656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830308.045466:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830308.045468:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830308.045470:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830308.045473:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4675 00000100:00080000:0.0:1787830308.045476:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830308.045479:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830308.045483:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830308.045492:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477062784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830308.045557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830308.045596:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830308.045615:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830308/real 1787830308] req@ffff9a7890a3de00 x1874671477062528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830308.045629:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477062528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830308.045652:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830308.045656:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830308.045659:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830308.045665:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830308.045687:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830308.045697:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830308/real 1787830308] req@ffff9a7890a3f480 x1874671477062656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830308.045706:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477062656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830308.045724:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830308.045728:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830308.045731:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830308.045737:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830308.045754:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830308.045763:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830308/real 1787830308] req@ffff9a7890a3da40 x1874671477062784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830308.045772:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477062784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830308.045781:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830308.045784:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830308.045787:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477062784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830309.066051:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830310.089969:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830311.114252:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830312.137968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830313.162032:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830313.162041:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830313.162045:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830313.162057:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4680 00000100:00080000:0.0:1787830313.162068:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830313.162075:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830313.162113:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477062912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830313.162135:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830313.162139:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830313.162157:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830313.162159:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830313.162163:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4680 00000100:00080000:0.0:1787830313.162167:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830313.162172:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830313.162181:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830313.162190:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477062912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830313.162195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477063040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830313.162205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830313.162208:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830313.162209:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830313.162221:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4680 00000100:00080000:0.0:1787830313.162225:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830313.162227:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830313.162231:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830313.162241:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477063168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830313.162250:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830313.162253:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830313.162254:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830313.162257:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4680 00000800:00000100:3.0:1787830313.162291:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830313.162469:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830313.162475:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830313.162511:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830313/real 1787830313] req@ffff9a7890a3c3c0 x1874671477062912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830313.162530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477062912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830313.162539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830313.162543:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830313.162547:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477062912:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787830313.162592:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830313.162608:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477063296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830313.162617:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830313.162622:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:3.0:1787830313.162623:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063040:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830313.162625:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830313.162634:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4680 00000100:00080000:0.0:1787830313.162639:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830313.162643:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830313.162648:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787830313.162657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830313.162662:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477063424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830313.162674:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830313/real 1787830313] req@ffff9a7890a3da40 x1874671477063040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830313.162691:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477063040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830313.162699:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830313.162703:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830313.162706:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830313.162711:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830313.162730:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830313.162741:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830313/real 1787830313] req@ffff9a7890a3f480 x1874671477063168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830313.162753:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477063168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830313.162760:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830313.162763:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830313.162766:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830313.162772:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830313.162789:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830313.162800:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830313/real 1787830313] req@ffff9a7890a3de00 x1874671477063296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830313.162811:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477063296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830313.162820:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830313.162822:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830313.162825:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830313.162861:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830313.162882:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830313.162893:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830313/real 1787830313] req@ffff9a7890a3fc00 x1874671477063424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830313.162910:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477063424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830313.162919:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830313.162922:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830313.162924:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830314.187017:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830315.210016:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830316.236264:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830317.258025:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830318.281986:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830318.282441:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830318.282448:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830318.282452:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830318.282462:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4685 00000100:00080000:2.0:1787830318.282472:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830318.282479:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830318.282517:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477063552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830318.282542:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830318.282546:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830318.282555:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830318.282560:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4685 00000100:00080000:2.0:1787830318.282564:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830318.282568:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830318.282577:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830318.282587:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477063552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830318.282599:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477063680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830318.282610:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830318.282613:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830318.282615:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830318.282668:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4685 00000100:00080000:2.0:1787830318.282675:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830318.282678:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063680:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830318.282683:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830318.282691:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830318.282706:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477063808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830318.282718:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830318.282722:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1787830318.282725:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063808:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830318.282726:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830318.282735:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4685 00000100:00080000:2.0:1787830318.282740:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830318.282744:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830318.282749:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830318.282761:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477063936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830318.282771:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830318.282775:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830318.282777:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830318.282782:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4685 00000100:00080000:2.0:1787830318.282785:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830318.282790:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830318.282794:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830318.282799:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830318.282804:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477064064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830318.282853:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830319.307096:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830320.331734:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830321.353983:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787830321.358177:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:1.0:1787830321.358217:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787830321.358231:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830321.358314:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830321.358386:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830321.358394:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:1.0:1787830321.358402:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787830321.359053:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830318/real 1787830321] req@ffff9a78841db0c0 x1874671477064064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.359076:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477064064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.359097:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.359102:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830321.359106:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064064:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830321.359114:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830318/real 1787830321] req@ffff9a78841da1c0 x1874671477063936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.359126:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477063936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.359134:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.359138:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830321.359141:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830321.359155:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830318/real 1787830321] req@ffff9a78841d9a40 x1874671477063808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.359166:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477063808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.359174:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.359177:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830321.359180:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830321.359186:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830318/real 1787830321] req@ffff9a78841d8780 x1874671477063680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.359196:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477063680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.359203:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.359207:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830321.359210:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477063680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830321.359216:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830318/real 1787830321] req@ffff9a78841d9e00 x1874671477063552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830321.359227:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477063552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830321.359234:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830321.359237:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830321.359241:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477063552:192.168.203.140@tcp:250:kworker.0 00010000:00010000:2.0:1787830322.383938:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830323.402030:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830323.402050:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830323.402057:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830323.402061:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830323.402069:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4690 00000100:00080000:0.0:1787830323.402089:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830323.402098:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830323.402129:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477064192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830323.402159:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830323.402163:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830323.402164:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830323.402168:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4690 00000100:00080000:0.0:1787830323.402171:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830323.402175:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830323.402183:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830323.402195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477064320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830323.402205:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830323.402208:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830323.402209:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830323.402212:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4690 00000100:00080000:0.0:1787830323.402215:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830323.402218:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830323.402222:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830323.402232:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477064448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830323.402241:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830323.402245:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830323.402247:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830323.402250:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4690 00000100:00080000:0.0:1787830323.402253:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830323.402256:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830323.402261:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830323.402277:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477064576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830323.402286:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830323.402289:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830323.402291:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830323.402294:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4690 00000100:00080000:0.0:1787830323.402297:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830323.402300:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830323.402304:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830323.402313:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477064704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830323.402408:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477064192:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830323.402463:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830323.402523:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830323/real 1787830323] req@ffff9a7890a3c000 x1874671477064192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830323.402540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477064192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830323.402552:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830323.402557:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830323.402562:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477064192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830323.402569:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830323.402591:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830323.402603:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830323/real 1787830323] req@ffff9a7890a3fc00 x1874671477064320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830323.402614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477064320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830323.402622:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830323.402625:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830323.402629:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830323.402634:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830323.402652:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830323.402663:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830323/real 1787830323] req@ffff9a7890a3de00 x1874671477064448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830323.402674:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477064448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830323.402681:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830323.402685:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830323.402688:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830323.402756:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830323.402784:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830323.402796:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830323/real 1787830323] req@ffff9a7890a3f480 x1874671477064576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830323.402806:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477064576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830323.402821:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830323.402824:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830323.402847:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830323.402855:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830323.402874:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830323.402885:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830323/real 1787830323] req@ffff9a7890a3da40 x1874671477064704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830323.402895:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477064704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830323.402904:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830323.402906:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830323.402909:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830324.425974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830325.451792:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830326.475313:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830327.497964:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830328.521992:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830328.523166:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830328.523174:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830328.523178:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830328.523187:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4696 00000100:00080000:2.0:1787830328.523196:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830328.523204:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830328.523239:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477064832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830328.523265:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830328.523268:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830328.523270:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830328.523274:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4696 00000100:00080000:2.0:1787830328.523277:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830328.523281:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830328.523290:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830328.523303:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477064960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830328.523313:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830328.523316:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830328.523318:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830328.523321:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4696 00000100:00080000:2.0:1787830328.523325:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830328.523328:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830328.523332:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830328.523342:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477065088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830328.523350:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830328.523353:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830328.523355:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830328.523358:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4696 00000100:00080000:2.0:1787830328.523361:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830328.523364:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830328.523369:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830328.523378:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477065216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830328.523386:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830328.523389:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830328.523391:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830328.523394:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4696 00000100:00080000:2.0:1787830328.523398:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830328.523401:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830328.523405:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830328.523414:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477065344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830328.524100:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477064832:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830328.524159:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830328.524210:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830328/real 1787830328] req@ffff9a78841dbc00 x1874671477064832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830328.524227:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477064832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830328.524235:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830328.524239:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830328.524242:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477064832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830328.524250:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830328.524274:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830328.524286:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830328/real 1787830328] req@ffff9a78841d9e00 x1874671477064960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830328.524298:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477064960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830328.524305:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830328.524309:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830328.524312:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477064960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830328.524318:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830328.524338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830328.524349:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830328/real 1787830328] req@ffff9a78841d8780 x1874671477065088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830328.524362:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477065088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830328.524369:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830328.524372:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830328.524375:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830328.524381:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830328.524400:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830328.524411:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830328/real 1787830328] req@ffff9a78841d9a40 x1874671477065216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830328.524427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477065216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830328.524436:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830328.524439:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830328.524442:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830328.524447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830328.524466:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830328.524476:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830328/real 1787830328] req@ffff9a78841da1c0 x1874671477065344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830328.524487:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477065344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830328.524495:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830328.524497:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830328.524500:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830329.545998:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830330.569978:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830331.594908:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830332.618374:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830333.642786:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830333.642793:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830333.642797:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830333.642805:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4701 00000100:00080000:0.0:1787830333.642813:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830333.642818:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830333.642883:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477065472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830333.642907:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830333.642910:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830333.642912:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830333.642915:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4701 00000100:00080000:0.0:1787830333.642918:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830333.642922:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830333.642931:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830333.642944:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477065600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830333.642952:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830333.642955:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830333.642957:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830333.642960:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4701 00000100:00080000:0.0:1787830333.642962:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830333.642965:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830333.642969:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830333.642978:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477065728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830333.642986:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830333.642989:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830333.642990:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830333.642994:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4701 00000100:00080000:0.0:1787830333.642997:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830333.643009:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830333.643015:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830333.643026:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477065856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830333.643035:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830333.643038:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830333.643039:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830333.643043:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4701 00000100:00080000:0.0:1787830333.643045:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830333.643048:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830333.643053:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830333.643062:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477065984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830333.644322:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477065472:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830333.644384:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830333.644431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830333/real 1787830333] req@ffff9a7890a3c3c0 x1874671477065472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830333.644448:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477065472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830333.644459:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830333.644463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830333.644472:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477065472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830333.644480:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830333.644502:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830333.644514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830333/real 1787830333] req@ffff9a7890a3da40 x1874671477065600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830333.644533:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477065600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830333.644540:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830333.644543:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830333.644546:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830333.644552:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830333.644571:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830333.644582:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830333/real 1787830333] req@ffff9a7890a3f480 x1874671477065728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830333.644592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477065728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830333.644599:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830333.644603:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830333.644606:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830333.644611:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830333.644629:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830333.644640:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830333/real 1787830333] req@ffff9a7890a3de00 x1874671477065856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830333.644649:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477065856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830333.644657:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830333.644660:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830333.644663:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830333.644668:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830333.644685:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830333.644695:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830333/real 1787830333] req@ffff9a7890a3fc00 x1874671477065984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830333.644705:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477065984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830333.644714:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830333.644716:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830333.644719:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477065984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830333.644781:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830334.666537:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830335.690550:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830336.713951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830337.738670:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830338.761976:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830338.761984:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830338.761988:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830338.761996:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4706 00000100:00080000:2.0:1787830338.762005:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830338.762010:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830338.762041:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477066112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830338.762060:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830338.762062:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830338.762064:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830338.762067:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4706 00000100:00080000:2.0:1787830338.762070:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830338.762074:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830338.762083:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830338.762094:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477066240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830338.762102:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830338.762105:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830338.762106:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830338.762110:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4706 00000100:00080000:2.0:1787830338.762113:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830338.762117:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830338.762121:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830338.762131:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477066368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830338.762137:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830338.762139:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830338.762140:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830338.762142:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4706 00000100:00080000:2.0:1787830338.762144:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830338.762146:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830338.762149:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830338.762155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477066496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830338.762163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830338.762166:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830338.762168:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830338.762173:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4706 00000100:00080000:2.0:1787830338.762176:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830338.762180:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830338.762185:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830338.762195:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477066624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830338.762205:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477066112:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830338.762276:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830338.762323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830338/real 1787830338] req@ffff9a78841db0c0 x1874671477066112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830338.762339:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477066112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830338.762349:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830338.762353:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830338.762358:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477066112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830338.762364:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830338.762385:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830338.762398:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830338/real 1787830338] req@ffff9a78841d83c0 x1874671477066240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830338.762408:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477066240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830338.762416:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830338.762419:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830338.762435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830338.762442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830338.762460:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830338.762472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830338/real 1787830338] req@ffff9a78841db480 x1874671477066368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830338.762489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477066368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830338.762497:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830338.762500:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830338.762503:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830338.762521:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830338.762541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830338.762554:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830338/real 1787830338] req@ffff9a78841da580 x1874671477066496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830338.763524:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477066496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830338.763539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830338.763545:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830338.763549:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830338.763579:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830338.763621:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830338.763635:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830338/real 1787830338] req@ffff9a78841d8000 x1874671477066624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830338.763647:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477066624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830338.763662:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830338.763665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830338.763668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830338.763807:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830339.785999:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830340.810314:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830341.834139:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830342.858338:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830343.881915:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830343.881924:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830343.881940:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830343.881950:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4711 00000100:00080000:2.0:1787830343.881959:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830343.881967:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787830343.881983:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830343.882002:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477066752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830343.882029:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830343.882033:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830343.882034:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830343.882038:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4711 00000100:00080000:2.0:1787830343.882041:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830343.882045:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830343.882053:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830343.882065:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477066880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830343.882079:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830343.882082:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830343.882084:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830343.882088:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4711 00000100:00080000:2.0:1787830343.882091:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830343.882094:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830343.882098:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830343.882107:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477067008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830343.882115:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830343.882118:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830343.882119:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830343.882128:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4711 00000100:00080000:2.0:1787830343.882131:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830343.882135:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830343.882140:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830343.882151:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477067136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830343.882159:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830343.882162:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830343.882164:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830343.882167:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4711 00000100:00080000:2.0:1787830343.882170:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830343.882173:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830343.882177:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830343.882194:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477067264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830343.882360:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477066752:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830343.882420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830343.882463:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830343/real 1787830343] req@ffff9a78841d92c0 x1874671477066752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830343.882479:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477066752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830343.882499:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830343.882504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830343.882508:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477066752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830343.882514:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830343.882535:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830343.882547:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830343/real 1787830343] req@ffff9a78841db840 x1874671477066880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830343.882558:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477066880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830343.882565:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830343.882569:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830343.882572:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477066880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830343.882577:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830343.882595:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830343.882606:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830343/real 1787830343] req@ffff9a78841d9680 x1874671477067008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830343.882616:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477067008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830343.882624:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830343.882626:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830343.882629:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830343.882635:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830343.882649:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830343.882669:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830343/real 1787830343] req@ffff9a78841d8b40 x1874671477067136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830343.882680:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477067136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830343.882824:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830343.882828:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830343.882851:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830343.882857:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830343.882880:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830343.882892:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830343/real 1787830343] req@ffff9a78841da940 x1874671477067264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830343.882902:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477067264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830343.882911:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830343.882914:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830343.882917:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830344.906229:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830345.930135:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830346.954206:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830347.977987:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830349.001934:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787830349.001958:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830349.002538:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830349.002544:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830349.002555:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4716 00000100:00080000:2.0:1787830349.002566:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830349.002573:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830349.002610:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477067392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830349.002645:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830349.002648:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830349.002650:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830349.002654:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4716 00000100:00080000:2.0:1787830349.002658:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830349.002663:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830349.002672:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830349.002687:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477067520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830349.002690:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477067392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830349.002705:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830349.002711:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830349.002713:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830349.002718:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4716 00000100:00080000:2.0:1787830349.002722:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830349.002726:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830349.002732:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830349.002745:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477067648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830349.002754:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830349.002758:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830349.002760:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830349.002766:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4716 00000800:00000100:3.0:1787830349.002766:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830349.002772:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830349.002777:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830349.002783:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830349.002795:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477067776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830349.002804:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830349.002808:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830349.002810:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830349.002824:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4716 00000100:00100000:3.0:1787830349.002828:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830349/real 1787830349] req@ffff9a78841da940 x1874671477067392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830349.002865:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830349.002869:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477067392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830349.002881:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:3.0:1787830349.002882:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:2.0:1787830349.002890:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:3.0:1787830349.002891:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830349.002911:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477067392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830349.002918:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477067904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830349.002938:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830349.002962:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830349.002975:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830349/real 1787830349] req@ffff9a78841d8b40 x1874671477067520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830349.002987:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477067520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830349.002995:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830349.002998:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830349.003002:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830349.003008:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830349.003032:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830349.003043:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830349/real 1787830349] req@ffff9a78841d9680 x1874671477067648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830349.003055:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477067648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830349.003062:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830349.003065:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830349.003068:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830349.003074:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830349.003093:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830349.003104:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830349/real 1787830349] req@ffff9a78841db840 x1874671477067776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830349.003115:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477067776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830349.003131:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830349.003134:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830349.003137:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830349.003142:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830349.003161:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830349.003172:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830349/real 1787830349] req@ffff9a78841d92c0 x1874671477067904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830349.003183:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477067904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830349.003192:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830349.003195:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830349.003198:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477067904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830350.026415:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830351.050065:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830352.073953:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830353.098764:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830354.121946:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830354.121956:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830354.121961:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830354.121970:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4721 00000100:00080000:2.0:1787830354.121980:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830354.121988:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830354.122025:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477068032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830354.122061:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830354.122065:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830354.122067:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830354.122071:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4721 00000100:00080000:2.0:1787830354.122074:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830354.122078:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830354.122087:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830354.122100:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477068160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830354.122108:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830354.122111:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830354.122113:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830354.122117:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4721 00000100:00080000:2.0:1787830354.122121:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830354.122125:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830354.122129:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830354.122139:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477068288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830354.122146:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830354.122150:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830354.122151:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830354.122155:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4721 00000100:00080000:2.0:1787830354.122157:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830354.122160:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830354.122165:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830354.122174:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477068416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830354.122182:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830354.122184:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830354.122186:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830354.122189:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4721 00000100:00100000:3.0:1787830354.122191:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477068032:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830354.122192:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830354.122195:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830354.122199:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830354.122208:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477068544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830354.122246:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830354.122323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830354/real 1787830354] req@ffff9a78841d8000 x1874671477068032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830354.122340:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477068032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830354.122359:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830354.122363:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830354.122367:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477068032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830354.122373:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830354.122394:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830354.122407:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830354/real 1787830354] req@ffff9a78841da580 x1874671477068160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830354.122417:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477068160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830354.122425:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830354.122428:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830354.122431:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830354.122437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830354.122455:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830354.122466:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830354/real 1787830354] req@ffff9a78841db480 x1874671477068288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830354.122476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477068288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830354.122484:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830354.122487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830354.122490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830354.122495:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830354.122524:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830354.122536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830354/real 1787830354] req@ffff9a78841d83c0 x1874671477068416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830354.122546:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477068416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830354.122554:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830354.122557:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830354.122560:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830354.122583:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830354.122600:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830354.122608:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830354/real 1787830354] req@ffff9a78841db0c0 x1874671477068544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830354.122614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477068544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830354.122620:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830354.122622:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830354.122624:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830354.122749:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830355.146241:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830356.169998:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830357.194053:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830358.217981:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830359.242242:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830359.242251:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830359.242256:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830359.242267:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4726 00000100:00080000:1.0:1787830359.242277:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830359.242283:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830359.242321:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477068672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830359.242326:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830359.242449:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477068672:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830359.242514:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830359.242543:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830359.242548:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830359.242550:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830359.242554:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4726 00000100:00080000:1.0:1787830359.242558:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830359.242563:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830359.242572:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830359.242585:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477068800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830359.242727:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830359/real 1787830359] req@ffff9a7890b1da40 x1874671477068672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830359.242749:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477068672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830359.242759:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830359.242763:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830359.242766:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477068672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830359.242790:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830359.242812:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830359.242864:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830359/real 1787830359] req@ffff9a7890b1d680 x1874671477068800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830359.242876:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477068800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830359.242884:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830359.242887:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830359.242890:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068800:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830359.243155:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830359.243159:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830359.243161:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830359.243166:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4726 00000100:00080000:1.0:1787830359.243170:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830359.243174:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830359.243179:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830359.243192:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477068928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830359.243237:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830359.243260:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830359.243282:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830359.243286:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830359.243288:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830359.243292:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4726 00000100:00080000:1.0:1787830359.243296:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830359.243300:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830359.243306:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830359.243317:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477069056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830359.243413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830359/real 1787830359] req@ffff9a7890b1de00 x1874671477068928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830359.243425:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477068928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830359.243438:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830359.243441:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830359.243445:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477068928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830359.243520:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830359.243543:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830359.243555:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830359/real 1787830359] req@ffff9a7890b1e580 x1874671477069056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830359.243566:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477069056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830359.243575:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830359.243578:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830359.243581:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069056:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787830359.243665:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830359.243669:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830359.243671:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830359.243675:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4726 00000100:00080000:1.0:1787830359.243680:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830359.243684:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830359.243689:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830359.243766:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477069184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830359.243810:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830359.243875:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830359.244415:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830359/real 1787830359] req@ffff9a7890b1e940 x1874671477069184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830359.244428:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477069184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830359.244437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830359.244440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830359.244444:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830360.266064:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830361.290041:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830362.314968:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830363.338073:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830364.362058:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830364.362068:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830364.362073:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830364.362083:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4731 00000100:00080000:2.0:1787830364.362096:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830364.362104:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830364.362152:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477069312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830364.362187:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830364.362191:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830364.362193:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830364.362197:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4731 00000100:00080000:2.0:1787830364.362206:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787830364.362206:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830364.362215:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830364.362226:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830364.362242:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477069440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830364.362243:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477069312:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830364.362259:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830364.362262:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830364.362263:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830364.362267:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4731 00000100:00080000:2.0:1787830364.362270:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830364.362274:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830364.362278:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830364.362289:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477069568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830364.362298:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830364.362301:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830364.362303:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830364.362307:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4731 00000100:00080000:2.0:1787830364.362310:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830364.362316:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787830364.362317:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787830364.362321:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830364.362331:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477069696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830364.362340:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830364.362343:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830364.362345:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830364.362349:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4731 00000100:00080000:2.0:1787830364.362354:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830364.362358:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830364.362363:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830364.362366:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830364/real 1787830364] req@ffff9a78841da1c0 x1874671477069312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830364.362376:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477069824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830364.362388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477069312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830364.362401:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830364.362406:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830364.362411:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477069312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830364.362433:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830364.362451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830364.362462:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830364/real 1787830364] req@ffff9a78841d9a40 x1874671477069440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830364.362473:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477069440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830364.362480:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830364.362483:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830364.362486:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830364.362491:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830364.362505:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830364.362513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830364/real 1787830364] req@ffff9a78841d8780 x1874671477069568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830364.362523:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477069568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830364.362530:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830364.362533:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830364.362535:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830364.362540:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830364.362555:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830364.362564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830364/real 1787830364] req@ffff9a78841d9e00 x1874671477069696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830364.362573:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477069696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830364.362581:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830364.362584:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830364.362587:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830364.362592:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830364.362611:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830364.362621:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830364/real 1787830364] req@ffff9a78841dbc00 x1874671477069824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830364.362631:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477069824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830364.362639:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830364.362642:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830364.362645:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477069824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830365.386021:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830366.409980:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830367.433969:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830368.458150:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830369.482162:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830369.482177:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830369.482183:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830369.482195:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4736 00000100:00080000:0.0:1787830369.482208:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830369.482217:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787830369.482240:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830369.482256:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477069952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830369.482332:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477069952:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830369.482403:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830369.482438:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830369.482442:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830369.482444:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830369.482447:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4736 00000100:00080000:0.0:1787830369.482451:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830369.482455:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830369.482464:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830369.482478:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477070080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830369.482544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830369/real 1787830369] req@ffff9a7890a3c000 x1874671477069952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830369.482561:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477069952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830369.482571:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830369.482575:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830369.482579:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477069952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830369.482605:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830369.482628:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830369.482641:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830369/real 1787830369] req@ffff9a7890a3fc00 x1874671477070080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830369.482652:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477070080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830369.482661:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830369.482664:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830369.482667:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830369.482720:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830369.482724:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830369.482726:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830369.482731:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4736 00000100:00080000:0.0:1787830369.482736:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830369.482740:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830369.482745:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830369.482767:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477070208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830369.482787:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830369.482790:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830369.482792:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830369.482796:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4736 00000100:00080000:0.0:1787830369.482800:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830369.482803:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830369.482808:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830369.482818:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477070336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830369.482827:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830369.482858:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830369.482859:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830369.482864:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4736 00000100:00080000:0.0:1787830369.482867:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830369.482871:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830369.482875:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830369.482895:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477070464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830369.482957:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830369.482992:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830369.483004:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830369/real 1787830369] req@ffff9a7890a3de00 x1874671477070208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830369.483017:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477070208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830369.483024:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830369.483027:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830369.483030:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830369.483035:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830369.483053:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830369.483075:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830369/real 1787830369] req@ffff9a7890a3f480 x1874671477070336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830369.483086:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477070336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830369.483095:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830369.483098:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830369.483101:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830369.483106:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830369.483122:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830369.483131:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830369/real 1787830369] req@ffff9a7890a3da40 x1874671477070464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830369.483142:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477070464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830369.483150:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830369.483153:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830369.483157:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830370.506087:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830371.530199:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830372.554021:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830373.578056:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830374.603162:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830374.603176:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830374.603183:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830374.603196:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4742 00000100:00080000:2.0:1787830374.603210:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830374.603218:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830374.603271:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477070592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830374.603394:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830374.603400:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830374.603402:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830374.603408:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4742 00000100:00080000:2.0:1787830374.603413:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830374.603418:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830374.603433:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830374.603449:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477070720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830374.603459:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830374.603463:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830374.603465:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830374.603471:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4742 00000100:00080000:2.0:1787830374.603475:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830374.603480:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830374.603485:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830374.603500:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477070848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830374.603509:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477070592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830374.603517:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830374.603523:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830374.603525:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830374.603531:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4742 00000100:00080000:2.0:1787830374.603534:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830374.603538:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830374.603544:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830374.603559:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477070976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830374.603570:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830374.603574:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830374.603576:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830374.603581:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4742 00000100:00080000:2.0:1787830374.603586:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830374.603594:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787830374.603594:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787830374.603604:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830374.603619:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477071104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830374.603660:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830374/real 1787830374] req@ffff9a78841dbc00 x1874671477070592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830374.603693:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477070592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830374.603705:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830374.603710:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830374.603714:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477070592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830374.603732:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830374.603757:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830374.603772:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830374/real 1787830374] req@ffff9a78841d9e00 x1874671477070720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830374.603783:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477070720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830374.603791:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830374.603794:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830374.603809:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830374.603859:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830374.603880:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830374.603892:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830374/real 1787830374] req@ffff9a78841d8780 x1874671477070848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830374.603903:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477070848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830374.603911:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830374.603914:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830374.603917:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830374.603923:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830374.603942:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830374.603953:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830374/real 1787830374] req@ffff9a78841d9a40 x1874671477070976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830374.603963:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477070976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830374.603971:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830374.603973:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830374.603976:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477070976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830374.603982:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830374.603999:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830374.604009:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830374/real 1787830374] req@ffff9a78841da1c0 x1874671477071104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830374.604020:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477071104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830374.604030:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830374.604033:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830374.604036:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830374.604508:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830375.628703:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830376.650727:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830377.674332:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830378.698042:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830379.722019:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830379.722430:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830379.722437:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830379.722441:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830379.722451:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4747 00000100:00080000:1.0:1787830379.722460:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830379.722467:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830379.722503:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477071232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830379.722526:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830379.722530:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830379.722532:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830379.722536:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4747 00000100:00080000:1.0:1787830379.722540:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830379.722545:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830379.722555:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830379.722560:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477071232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830379.722570:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477071360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830379.722588:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830379.722591:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830379.722593:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830379.722598:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4747 00000100:00080000:1.0:1787830379.722602:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787830379.722607:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830379.722608:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830379.722616:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830379.722629:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477071488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830379.722639:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830379.722643:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830379.722645:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787830379.722648:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830379/real 1787830379] req@ffff9a7890b1e940 x1874671477071232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830379.722652:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4747 00000100:00080000:1.0:1787830379.722659:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830379.722665:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830379.722670:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477071232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1787830379.722671:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:3.0:1787830379.722692:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830379.722697:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830379.722705:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477071616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830379.722708:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477071232:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830379.722721:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830379.722724:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830379.722726:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830379.722731:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4747 00000100:00080000:1.0:1787830379.722734:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830379.722738:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830379.722742:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830379.722748:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830379.722755:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477071744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830379.722781:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830379.722806:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830379/real 1787830379] req@ffff9a7890b1e580 x1874671477071360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830379.722818:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477071360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830379.722825:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830379.722845:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830379.722850:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830379.722855:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830379.722875:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830379.722887:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830379/real 1787830379] req@ffff9a7890b1de00 x1874671477071488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830379.722903:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477071488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830379.722910:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830379.722913:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830379.722916:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830379.722922:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830379.722940:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830379.722950:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830379/real 1787830379] req@ffff9a7890b1d680 x1874671477071616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830379.722967:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477071616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830379.722976:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830379.722979:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830379.722982:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830379.723056:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830379.723076:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830379.723087:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830379/real 1787830379] req@ffff9a7890b1da40 x1874671477071744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830379.723098:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477071744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830379.723108:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830379.723110:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830379.723113:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477071744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830380.746417:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830381.770968:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830382.793974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830383.818058:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830384.842221:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830384.842467:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830384.842474:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830384.842478:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830384.842488:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4752 00000100:00080000:0.0:1787830384.842497:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830384.842503:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830384.842661:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477071872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830384.842689:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830384.842693:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830384.842696:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830384.842700:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4752 00000100:00080000:0.0:1787830384.842703:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830384.842708:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830384.842716:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830384.842729:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477072000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830384.842738:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830384.842740:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830384.842742:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830384.842746:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4752 00000100:00080000:0.0:1787830384.842748:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830384.842751:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830384.842755:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830384.842764:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477072128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830384.842771:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830384.842774:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830384.842776:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830384.842780:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4752 00000100:00080000:0.0:1787830384.842783:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830384.842787:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830384.842802:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830384.842813:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477072256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830384.842821:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830384.842824:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830384.842826:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830384.842853:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4752 00000100:00080000:0.0:1787830384.842857:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830384.842861:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830384.842865:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830384.842876:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477072384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830384.842999:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477071872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830384.843088:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830384.843111:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830384.843127:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830384.843146:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830385.865988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830386.890853:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830387.914118:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787830387.916393:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787830387.916439:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787830387.916454:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830387.916491:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830387.916501:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830387.916508:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830387.916515:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:3.0:1787830387.916690:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830384/real 1787830387] req@ffff9a7890a3c3c0 x1874671477071872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830387.916714:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477071872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830387.916727:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830387.916732:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830387.916736:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477071872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830387.916744:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830384/real 1787830387] req@ffff9a7890a3da40 x1874671477072000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830387.916755:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477072000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830387.916762:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830387.916766:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830387.916769:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830387.916776:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830384/real 1787830387] req@ffff9a7890a3f480 x1874671477072128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830387.916786:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477072128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830387.916793:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830387.916797:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830387.916799:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830387.916805:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830384/real 1787830387] req@ffff9a7890a3de00 x1874671477072256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830387.916815:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477072256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830387.916823:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830387.916826:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830387.916963:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830387.916971:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830384/real 1787830387] req@ffff9a7890a3fc00 x1874671477072384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830387.916985:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477072384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830387.916994:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830387.916997:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830387.917000:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830388.937960:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830389.962691:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830389.962701:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830389.962704:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830389.962716:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4757 00000100:00080000:1.0:1787830389.962726:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830389.962733:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830389.962766:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477072512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830389.962828:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477072512:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830389.962912:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830389.963006:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830389.963011:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830389.963013:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830389.963017:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4757 00000100:00080000:1.0:1787830389.963021:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830389.963026:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830389.963037:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830389.963051:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477072640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830389.964367:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830389/real 1787830389] req@ffff9a7890b1f0c0 x1874671477072512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830389.964397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477072512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830389.964406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830389.964410:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830389.964414:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477072512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830389.964442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830389.964484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830389.964499:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830389/real 1787830389] req@ffff9a7890b1f840 x1874671477072640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830389.964510:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477072640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830389.964518:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830389.964521:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830389.964525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072640:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787830389.964608:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830389.964667:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830389.964672:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830389.964675:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830389.964679:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4757 00000100:00080000:1.0:1787830389.964684:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830389.964689:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830389.964697:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830389.964715:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477072768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830389.964763:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830389.964786:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830389.964806:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830389.964810:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830389.964812:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830389.964816:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4757 00000100:00080000:1.0:1787830389.964821:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830389.964825:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830389.964843:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830389.964856:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477072896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830389.965709:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830389/real 1787830389] req@ffff9a7890b1c000 x1874671477072768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830389.965727:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477072768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830389.965745:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830389.965749:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830389.965753:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830389.965773:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830389.965803:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830389.965814:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830389/real 1787830389] req@ffff9a7890b1f480 x1874671477072896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830389.965826:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477072896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830389.965848:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830389.965851:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830389.965854:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477072896:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787830389.966080:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830389.966085:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830389.966088:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830389.966092:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4757 00000100:00080000:1.0:1787830389.966097:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830389.966102:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830389.966108:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830389.966126:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477073024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830389.966167:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830389.966189:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830389.966415:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830389/real 1787830389] req@ffff9a7890b1fc00 x1874671477073024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830389.966429:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477073024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830389.966437:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830389.966440:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830389.966443:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830390.986197:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830392.010042:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830393.034452:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830394.058020:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830395.081976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830395.082011:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830395.082074:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830395.082079:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830395.082088:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4762 00000100:00080000:1.0:1787830395.082098:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830395.082104:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830395.082142:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477073152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830395.082168:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830395.082172:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830395.082174:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830395.082177:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4762 00000100:00080000:1.0:1787830395.082181:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830395.082186:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830395.082194:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830395.082206:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477073280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830395.082215:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830395.082218:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830395.082220:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830395.082224:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4762 00000100:00080000:1.0:1787830395.082227:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830395.082231:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830395.082235:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830395.082244:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477073408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830395.082253:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830395.082256:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830395.082258:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830395.082268:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4762 00000100:00080000:1.0:1787830395.082272:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830395.082276:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830395.082281:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830395.082295:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477073536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830395.082295:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477073152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830395.082312:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830395.082316:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830395.082318:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830395.082322:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4762 00000100:00080000:1.0:1787830395.082326:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830395.082330:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830395.082335:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830395.082345:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477073664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830395.082358:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830395.082405:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830395/real 1787830395] req@ffff9a7890b1fc00 x1874671477073152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830395.082422:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477073152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830395.082432:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830395.082436:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830395.082440:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477073152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830395.082447:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830395.082469:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830395.082482:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830395/real 1787830395] req@ffff9a7890b1f480 x1874671477073280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830395.082494:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477073280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830395.082502:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830395.082504:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830395.082508:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830395.082513:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830395.082532:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830395.082543:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830395/real 1787830395] req@ffff9a7890b1c000 x1874671477073408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830395.082554:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477073408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830395.082562:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830395.082565:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830395.082568:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830395.082603:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830395.082623:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830395.082634:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830395/real 1787830395] req@ffff9a7890b1f840 x1874671477073536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830395.082645:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477073536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830395.082654:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830395.082657:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830395.082660:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830395.082666:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830395.082683:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830395.082695:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830395/real 1787830395] req@ffff9a7890b1f0c0 x1874671477073664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830395.082713:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477073664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830395.082721:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830395.082724:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830395.082727:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830396.106039:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830397.130200:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830398.154045:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830399.178734:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830400.201915:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830400.201925:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830400.201929:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830400.201937:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4767 00000100:00080000:1.0:1787830400.201948:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830400.201954:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830400.201999:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477073792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830400.202022:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830400.202026:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830400.202028:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830400.202033:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4767 00000100:00080000:1.0:1787830400.202037:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830400.202041:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830400.202049:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787830400.202050:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830400.202063:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477073920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830400.202073:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830400.202076:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830400.202078:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830400.202082:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4767 00000100:00080000:1.0:1787830400.202086:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830400.202090:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830400.202105:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830400.202106:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477073792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830400.202117:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477074048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830400.202125:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830400.202128:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830400.202129:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830400.202132:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4767 00000100:00080000:1.0:1787830400.202135:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830400.202138:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830400.202143:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830400.202151:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477074176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830400.202159:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830400.202162:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830400.202164:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830400.202167:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4767 00000800:00000100:3.0:1787830400.202202:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830400.202348:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830400.202354:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830400.202358:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830400.202370:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477074304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830400.202385:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830400/real 1787830400] req@ffff9a7890b1da40 x1874671477073792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830400.202400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477073792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830400.202410:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830400.202414:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830400.202417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477073792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830400.202435:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830400.202452:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830400.202460:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830400/real 1787830400] req@ffff9a7890b1d680 x1874671477073920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830400.202467:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477073920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830400.202476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830400.202478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830400.202480:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477073920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830400.202483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830400.202492:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830400.202497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830400/real 1787830400] req@ffff9a7890b1de00 x1874671477074048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830400.202503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477074048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830400.202508:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830400.202509:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830400.202511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830400.202514:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830400.202522:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830400.202527:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830400/real 1787830400] req@ffff9a7890b1e580 x1874671477074176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830400.202533:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477074176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830400.202550:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830400.202553:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830400.202556:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830400.202561:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830400.202575:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830400.202581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830400/real 1787830400] req@ffff9a7890b1e940 x1874671477074304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830400.202588:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477074304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830400.202601:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830400.202604:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830400.202607:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830401.225976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830402.249997:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830403.274414:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830404.298135:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830405.322081:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830405.322121:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830405.322125:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830405.322129:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830405.322139:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4772 00000100:00080000:1.0:1787830405.322159:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830405.322167:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830405.322209:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477074432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830405.322236:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830405.322242:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830405.322244:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830405.322248:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4772 00000100:00080000:1.0:1787830405.322252:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830405.322256:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830405.322265:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830405.322276:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477074560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830405.322284:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830405.322287:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830405.322289:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830405.322293:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4772 00000100:00080000:1.0:1787830405.322295:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830405.322298:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830405.322302:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830405.322311:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477074688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830405.322318:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830405.322322:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830405.322323:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830405.322326:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4772 00000100:00080000:1.0:1787830405.322329:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830405.322332:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830405.322337:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830405.322347:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477074816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830405.322354:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830405.322356:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830405.322358:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830405.322362:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4772 00000100:00080000:1.0:1787830405.322364:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830405.322368:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830405.322372:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830405.322381:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477074944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830405.322498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477074432:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830405.322563:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830405.322607:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830405/real 1787830405] req@ffff9a7890b1e940 x1874671477074432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830405.322622:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477074432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830405.322632:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830405.322636:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830405.322640:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477074432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830405.322646:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830405.322735:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830405.322747:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830405/real 1787830405] req@ffff9a7890b1e580 x1874671477074560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830405.322758:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477074560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830405.322766:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830405.322769:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830405.322772:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830405.322778:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830405.322806:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830405.322816:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830405/real 1787830405] req@ffff9a7890b1de00 x1874671477074688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830405.322827:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477074688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830405.322854:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830405.323716:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830405.323721:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830405.323727:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830405.323752:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830405.323762:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830405/real 1787830405] req@ffff9a7890b1d680 x1874671477074816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830405.323775:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477074816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830405.323782:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830405.323784:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830405.323786:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830405.323789:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830405.323799:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830405.323805:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830405/real 1787830405] req@ffff9a7890b1da40 x1874671477074944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830405.323811:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477074944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830405.323816:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830405.323818:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830405.323819:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477074944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830406.346157:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830407.369991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830408.394438:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830409.418320:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830410.442149:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830410.442159:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830410.442163:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830410.442172:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4777 00000100:00080000:0.0:1787830410.442183:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830410.442190:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830410.442223:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477075072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830410.442247:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830410.442250:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830410.442252:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830410.442256:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4777 00000100:00080000:0.0:1787830410.442260:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830410.442264:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830410.442273:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830410.442286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477075200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830410.442296:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830410.442299:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830410.442301:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830410.442305:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4777 00000100:00080000:0.0:1787830410.442308:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830410.442311:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830410.442316:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830410.442326:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477075328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830410.442335:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830410.442338:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830410.442340:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830410.442343:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4777 00000100:00080000:0.0:1787830410.442346:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830410.442349:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830410.442354:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830410.442363:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477075456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830410.442377:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830410.442380:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830410.442382:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830410.442386:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4777 00000100:00080000:0.0:1787830410.442389:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830410.442393:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830410.442397:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830410.442407:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477075584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830410.443065:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477075072:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830410.443113:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830410.443146:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830410/real 1787830410] req@ffff9a7890a3c000 x1874671477075072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830410.443157:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477075072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830410.443165:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830410.443168:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830410.443171:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477075072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830410.443175:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830410.443187:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830410.443193:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830410/real 1787830410] req@ffff9a7890a3fc00 x1874671477075200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830410.443200:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477075200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830410.443204:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830410.443206:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830410.443208:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830410.443211:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830410.443219:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830410.443224:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830410/real 1787830410] req@ffff9a7890a3de00 x1874671477075328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830410.443230:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477075328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830410.443234:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830410.443236:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830410.443238:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830410.443241:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830410.443249:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830410.443254:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830410/real 1787830410] req@ffff9a7890a3f480 x1874671477075456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830410.443260:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477075456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830410.443265:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830410.443266:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830410.443268:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830410.443271:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830410.443279:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830410.443284:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830410/real 1787830410] req@ffff9a7890a3da40 x1874671477075584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830410.443290:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477075584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830410.443295:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830410.443296:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830410.443298:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830410.443340:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830411.465981:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830412.490050:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830413.515747:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830414.538016:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830415.562019:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830415.562074:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830415.562080:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830415.562084:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830415.562094:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4783 00000100:00080000:2.0:1787830415.562105:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830415.562111:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830415.562143:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477075712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830415.562186:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477075712:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830415.562238:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830415.562274:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830415.562278:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830415.562279:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830415.562283:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4783 00000100:00080000:2.0:1787830415.562287:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830415.562291:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830415.562299:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830415.562310:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477075840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830415.562704:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830415/real 1787830415] req@ffff9a78841db0c0 x1874671477075712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830415.562721:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477075712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830415.562729:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830415.562732:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830415.562790:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477075712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830415.562810:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830415.562850:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830415.562864:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830415/real 1787830415] req@ffff9a78841d83c0 x1874671477075840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830415.562875:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477075840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830415.562883:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830415.562887:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830415.562890:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075840:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830415.563027:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830415.563030:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830415.563032:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830415.563035:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4783 00000100:00080000:2.0:1787830415.563038:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830415.563041:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830415.563045:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830415.563108:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477075968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830415.563144:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075968:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830415.563169:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787830415.563171:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830415.563173:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830415.563175:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830415.563179:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4783 00000100:00100000:3.0:1787830415.563181:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830415/real 1787830415] req@ffff9a78841db480 x1874671477075968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830415.563183:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830415.563187:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830415.563192:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830415.563199:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477075968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830415.563203:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477076096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830415.563208:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830415.563211:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00080000:2.0:1787830415.563212:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787830415.563214:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477075968:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830415.563215:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830415.563217:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787830415.563225:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830415.563242:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830415.563355:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830415/real 1787830415] req@ffff9a78841da580 x1874671477076096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830415.563367:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477076096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830415.563376:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830415.563378:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830415.563381:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076096:192.168.203.140@tcp:8:kworker.0 00000100:00080000:2.0:1787830415.563465:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4783 00000100:00080000:2.0:1787830415.563470:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830415.563474:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830415.563479:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830415.563491:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477076224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830415.563558:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830415.563579:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830415.563589:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830415/real 1787830415] req@ffff9a78841d8000 x1874671477076224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830415.563600:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477076224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830415.563613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830415.563616:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830415.563619:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830416.585947:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830417.609962:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830418.634158:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830419.657954:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830420.682025:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830420.682034:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830420.682038:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830420.682047:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4788 00000100:00080000:1.0:1787830420.682057:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830420.682063:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830420.682099:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477076352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830420.682130:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830420.682134:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830420.682136:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830420.682140:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4788 00000100:00080000:1.0:1787830420.682144:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830420.682147:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830420.682156:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830420.682168:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477076480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830420.682177:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830420.682180:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830420.682182:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830420.682188:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4788 00000100:00100000:0.0:1787830420.682189:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477076352:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830420.682192:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830420.682199:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830420.682204:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830420.682214:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477076608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830420.682225:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830420.682228:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830420.682230:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830420.682234:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4788 00000800:00000100:0.0:1787830420.682252:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830420.682364:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830420.682370:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830420.682377:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830420.682390:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477076736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830420.682412:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830420.682415:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830420.682417:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830420.682421:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4788 00000100:00080000:1.0:1787830420.682424:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830420.682428:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830420.682432:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830420.682442:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477076864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830420.682453:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830420/real 1787830420] req@ffff9a7890b1f0c0 x1874671477076352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830420.682469:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477076352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830420.682476:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830420.682479:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830420.682481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477076352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830420.682486:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830420.682503:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830420.682514:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830420/real 1787830420] req@ffff9a7890b1f840 x1874671477076480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830420.682527:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477076480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830420.682535:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830420.682538:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830420.682541:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830420.682562:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830420.682582:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830420.682593:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830420/real 1787830420] req@ffff9a7890b1c000 x1874671477076608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830420.682605:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477076608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830420.682612:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830420.682615:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830420.682618:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830420.682624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830420.682642:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830420.682652:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830420/real 1787830420] req@ffff9a7890b1f480 x1874671477076736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830420.682662:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477076736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830420.682675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830420.682678:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830420.682681:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830420.682686:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830420.682703:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830420.682713:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830420/real 1787830420] req@ffff9a7890b1fc00 x1874671477076864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830420.682723:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477076864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830420.682731:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830420.682733:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830420.682736:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477076864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830420.683162:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830421.706012:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830422.730024:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830423.754479:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830424.779916:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830425.802059:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830425.802070:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830425.802075:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830425.802085:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4793 00000100:00080000:0.0:1787830425.802096:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787830425.802100:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830425.802105:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830425.802142:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477076992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830425.802173:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830425.802177:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830425.802179:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830425.802183:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4793 00000100:00080000:0.0:1787830425.802186:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830425.802190:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830425.802199:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830425.802213:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477077120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830425.802222:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830425.802225:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830425.802227:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830425.802231:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4793 00000100:00080000:0.0:1787830425.802234:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830425.802237:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830425.802241:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830425.802251:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477077248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830425.802259:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830425.802263:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830425.802264:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830425.802268:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4793 00000100:00080000:0.0:1787830425.802270:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830425.802274:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830425.802279:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830425.802289:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477077376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830425.802297:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830425.802300:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830425.802302:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830425.802305:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4793 00000100:00080000:0.0:1787830425.802308:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830425.802312:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830425.802316:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830425.802326:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477077504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830425.802402:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477076992:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830425.802455:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830425.802497:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830425/real 1787830425] req@ffff9a7890a3c3c0 x1874671477076992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830425.802511:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477076992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830425.802521:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830425.802525:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830425.802528:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477076992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830425.802534:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830425.802556:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830425.802567:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830425/real 1787830425] req@ffff9a7890a3da40 x1874671477077120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830425.802577:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477077120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830425.802583:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830425.802586:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830425.802589:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830425.802594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830425.802611:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830425.802622:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830425/real 1787830425] req@ffff9a7890a3f480 x1874671477077248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830425.802631:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477077248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830425.802638:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830425.802703:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830425.802708:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830425.802714:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830425.802733:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830425.802745:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830425/real 1787830425] req@ffff9a7890a3de00 x1874671477077376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830425.802761:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477077376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830425.802771:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830425.802774:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830425.802776:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830425.802780:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830425.802797:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830425.802807:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830425/real 1787830425] req@ffff9a7890a3fc00 x1874671477077504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830425.802816:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477077504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830425.802824:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830425.802827:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830425.802920:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830426.826714:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830427.850060:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830428.873965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830429.898021:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830430.922014:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830430.922025:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830430.922030:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830430.922040:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4798 00000100:00080000:2.0:1787830430.922051:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830430.922057:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830430.922097:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477077632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830430.922154:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477077632:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830430.922218:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830430.922247:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830430.922250:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830430.922252:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830430.922257:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4798 00000100:00080000:2.0:1787830430.922260:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830430.922266:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830430.922278:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787830430.922292:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830430.922296:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477077760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830430.922391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830430/real 1787830430] req@ffff9a78841d92c0 x1874671477077632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830430.922409:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477077632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830430.922420:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830430.922424:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830430.922428:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477077632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830430.922455:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830430.922478:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830430.922492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830430/real 1787830430] req@ffff9a78841db840 x1874671477077760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830430.922505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477077760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830430.922512:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830430.922514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830430.922516:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077760:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830430.922560:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830430.922566:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830430.922567:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830430.922571:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4798 00000100:00080000:2.0:1787830430.922574:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830430.922577:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830430.922582:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830430.922593:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477077888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830430.922610:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830430.922615:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830430.922617:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787830430.922622:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077888:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830430.922624:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4798 00000100:00080000:2.0:1787830430.922631:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830430.922636:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830430.922642:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:0.0:1787830430.922657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830430.922658:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477078016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830430.922670:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830430.922675:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00100000:0.0:1787830430.922676:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830430/real 1787830430] req@ffff9a78841d9680 x1874671477077888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830430.922678:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830430.922687:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4798 00000100:00080000:2.0:1787830430.922693:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830430.922699:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830430.922704:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477077888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787830430.922704:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787830430.922717:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1787830430.922717:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477078144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830430.922727:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830430.922736:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477077888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830430.922752:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830430.922776:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830430.922788:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830430/real 1787830430] req@ffff9a78841d8b40 x1874671477078016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830430.922807:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477078016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830430.922818:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830430.922821:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830430.922825:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830430.922855:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830430.922876:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830430.922888:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830430/real 1787830430] req@ffff9a78841da940 x1874671477078144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830430.922909:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477078144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830430.922924:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830430.922928:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830430.922931:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830431.945974:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830432.973676:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830433.993999:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830435.018204:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830436.041964:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830436.041975:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830436.041979:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830436.041989:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4803 00000100:00080000:2.0:1787830436.042001:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830436.042008:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830436.042046:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477078272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830436.042085:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830436.042090:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830436.042092:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830436.042096:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4803 00000100:00080000:2.0:1787830436.042100:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830436.042104:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830436.042114:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830436.042129:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477078400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830436.042137:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830436.042140:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830436.042141:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830436.042145:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4803 00000100:00080000:2.0:1787830436.042147:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830436.042151:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830436.042155:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830436.042164:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477078528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830436.042172:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830436.042175:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830436.042178:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830436.042181:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4803 00000100:00080000:2.0:1787830436.042189:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830436.042194:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830436.042199:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477078272:192.168.203.140@tcp:250:kworker.0 00000008:00100000:2.0:1787830436.042200:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00010000:00010000:1.0:1787830436.042217:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830436.042217:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477078656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830436.042230:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830436.042233:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830436.042235:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830436.042239:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4803 00000100:00080000:2.0:1787830436.042257:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830436.042265:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787830436.042270:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:2.0:1787830436.042272:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830436.042286:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477078784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830436.042330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830436/real 1787830436] req@ffff9a78841da940 x1874671477078272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830436.042348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477078272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830436.042362:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830436.042366:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830436.042370:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477078272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830436.042376:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830436.042396:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830436.042408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830436/real 1787830436] req@ffff9a78841d8b40 x1874671477078400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830436.042418:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477078400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830436.042426:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830436.042429:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830436.042432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830436.042437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830436.042454:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830436.042465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830436/real 1787830436] req@ffff9a78841d9680 x1874671477078528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830436.042476:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477078528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830436.042483:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830436.042486:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830436.042489:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830436.042511:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830436.042530:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830436.042541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830436/real 1787830436] req@ffff9a78841db840 x1874671477078656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830436.042552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477078656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830436.042561:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830436.042564:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830436.042567:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830436.042572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830436.042590:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830436.042600:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830436/real 1787830436] req@ffff9a78841d92c0 x1874671477078784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830436.042611:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477078784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830436.042621:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830436.042623:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830436.042627:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477078784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830437.066349:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830438.090180:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830439.114807:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830440.138157:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830441.162042:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830441.162128:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830441.162136:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830441.162139:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830441.162149:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4808 00000100:00080000:1.0:1787830441.162158:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830441.162164:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830441.162196:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477078912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830441.162270:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477078912:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830441.162321:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830441.162635:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830441.162639:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830441.162641:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830441.162646:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4808 00000100:00080000:1.0:1787830441.162650:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830441.162665:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830441.162674:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830441.162675:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830441/real 1787830441] req@ffff9a7890b1fc00 x1874671477078912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830441.162692:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477079040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830441.162697:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477078912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830441.162712:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830441.162715:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830441.162717:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830441.162721:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830441.162724:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787830441.162728:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4808 00000100:00100000:0.0:1787830441.162733:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477078912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830441.162733:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830441.162740:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830441.162745:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830441.162758:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477079168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830441.162760:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079040:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830441.162775:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830441.162779:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830441.162780:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830441.162785:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4808 00000100:00080000:1.0:1787830441.162789:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830441.162793:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787830441.162797:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787830441.162800:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830441.162814:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477079296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830441.162816:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830441/real 1787830441] req@ffff9a7890b1f480 x1874671477079040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830441.162867:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477079040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830441.162877:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830441.162880:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830441.162884:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830441.162901:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830441.162920:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830441.162931:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830441/real 1787830441] req@ffff9a7890b1c000 x1874671477079168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830441.162944:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477079168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830441.162952:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830441.162959:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830441.162961:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830441.162966:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4808 00000100:00080000:1.0:1787830441.162971:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830441.162978:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:0.0:1787830441.162979:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000008:00100000:1.0:1787830441.162984:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787830441.162985:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830441.162992:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830441.163001:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477079424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830441.163008:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830441.163032:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830441.163043:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830441/real 1787830441] req@ffff9a7890b1f840 x1874671477079296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830441.163055:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477079296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830441.163064:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830441.163067:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830441.163070:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830441.163082:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830441.163099:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830441.163109:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830441/real 1787830441] req@ffff9a7890b1f0c0 x1874671477079424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830441.163119:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477079424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830441.163127:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830441.163129:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830441.163133:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830442.187476:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830443.210756:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830444.234052:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830445.258075:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830446.281960:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830446.281971:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830446.281975:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830446.281984:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4813 00000100:00080000:2.0:1787830446.281995:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830446.282002:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830446.282038:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477079552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830446.282058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830446.282063:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830446.282068:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830446.282070:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830446.282074:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4813 00000100:00080000:2.0:1787830446.282078:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830446.282082:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830446.282090:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830446.282103:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477079680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830446.282110:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477079552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830446.282115:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830446.282120:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830446.282121:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830446.282125:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4813 00000100:00080000:2.0:1787830446.282129:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830446.282133:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830446.282137:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830446.282150:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477079808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830446.282160:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830446.282164:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830446.282166:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787830446.282172:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830446.282172:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4813 00000100:00080000:2.0:1787830446.282179:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830446.282184:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830446.282189:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830446.282209:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477079936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830446.282215:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830446/real 1787830446] req@ffff9a78841d8000 x1874671477079552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830446.282233:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830446.282240:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830446.282244:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830446.282250:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4813 00000100:00100000:0.0:1787830446.282255:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477079552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830446.282255:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830446.282262:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830446.282270:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:0.0:1787830446.282274:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830446.282281:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830446.282285:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477080064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830446.282289:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477079552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830446.282317:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830446.282338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830446.282348:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830446/real 1787830446] req@ffff9a78841da580 x1874671477079680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830446.282359:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477079680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830446.282366:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830446.282369:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830446.282371:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830446.282376:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830446.282389:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830446.282396:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830446/real 1787830446] req@ffff9a78841db480 x1874671477079808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830446.282406:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477079808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830446.282412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830446.282414:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830446.282417:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830446.282421:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830446.282434:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830446.282442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830446/real 1787830446] req@ffff9a78841d83c0 x1874671477079936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830446.282453:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477079936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830446.282462:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830446.282465:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830446.282468:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477079936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830446.282474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830446.282491:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830446.282502:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830446/real 1787830446] req@ffff9a78841db0c0 x1874671477080064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830446.282512:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477080064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830446.282522:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830446.282524:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830446.282527:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830447.306056:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830448.330009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830449.354020:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830450.377965:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830451.402315:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830451.402325:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830451.402329:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830451.402340:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4818 00000100:00080000:1.0:1787830451.402350:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830451.402357:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830451.402398:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477080192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830451.402425:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830451.402430:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830451.402434:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830451.402436:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830451.402440:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4818 00000100:00080000:1.0:1787830451.402444:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830451.402457:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830451.402465:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830451.402476:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477080320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830451.402484:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830451.402487:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830451.402489:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830451.402492:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4818 00000100:00080000:1.0:1787830451.402496:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830451.402499:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830451.402504:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830451.402514:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477080448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830451.402522:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830451.402525:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830451.402527:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830451.402530:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4818 00000100:00080000:1.0:1787830451.402533:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830451.402536:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830451.402540:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830451.402547:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477080576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830451.402559:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830451.402562:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830451.402563:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830451.402566:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4818 00000100:00080000:1.0:1787830451.402569:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830451.402571:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830451.402575:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830451.402582:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477080704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830451.403245:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477080192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830451.403330:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830451.403348:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830451.403370:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830451.403394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830452.426227:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830453.450061:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787830454.474408:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787830454.474450:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830454.474465:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830454.474504:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830454.474512:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830454.474519:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830454.474526:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787830454.474599:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830451/real 1787830454] req@ffff9a7890b1da40 x1874671477080192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787830454.474644:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830454.474685:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477080192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830454.474696:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830454.474700:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830454.474704:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477080192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830454.474710:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830451/real 1787830454] req@ffff9a7890b1d680 x1874671477080320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830454.474721:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477080320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830454.474727:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830454.474730:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830454.474732:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830454.474737:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830451/real 1787830454] req@ffff9a7890b1de00 x1874671477080448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830454.474747:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477080448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830454.474753:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830454.474756:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830454.474758:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830454.474764:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830451/real 1787830454] req@ffff9a7890b1e580 x1874671477080576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830454.474774:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477080576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830454.474782:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830454.474784:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830454.474787:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830454.474792:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830451/real 1787830454] req@ffff9a7890b1e940 x1874671477080704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830454.474802:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477080704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830454.474811:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830454.474814:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830454.474817:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830455.498236:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830456.522009:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830456.522019:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830456.522023:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830456.522037:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4824 00000100:00080000:1.0:1787830456.522051:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830456.522060:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787830456.522098:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830456.522101:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477080832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830456.522136:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830456.522140:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830456.522141:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830456.522157:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4824 00000100:00080000:1.0:1787830456.522161:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830456.522165:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830456.522177:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830456.522190:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477080960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830456.522200:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830456.522203:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830456.522204:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830456.522208:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4824 00000100:00080000:1.0:1787830456.522212:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830456.522216:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830456.522221:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830456.522232:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477081088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830456.522241:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830456.522245:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830456.522246:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830456.522249:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4824 00000100:00080000:1.0:1787830456.522252:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830456.522255:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830456.522259:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830456.522267:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477081216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830456.522273:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830456.522276:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830456.522277:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830456.522280:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4824 00000100:00080000:1.0:1787830456.522281:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830456.522283:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830456.522286:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830456.522296:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477081344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830456.522558:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477080832:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830456.522635:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830456.522686:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830456/real 1787830456] req@ffff9a7890b1e940 x1874671477080832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830456.522705:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477080832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830456.522715:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830456.522720:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830456.522724:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477080832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830456.522732:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830456.522754:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830456.522766:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830456/real 1787830456] req@ffff9a7890b1e580 x1874671477080960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830456.522784:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477080960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830456.522792:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830456.522795:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830456.522797:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477080960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830456.522803:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830456.522821:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830456.522859:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830456/real 1787830456] req@ffff9a7890b1de00 x1874671477081088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830456.522879:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477081088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830456.522886:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830456.522890:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830456.522893:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830456.522899:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830456.522918:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830456.522930:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830456/real 1787830456] req@ffff9a7890b1d680 x1874671477081216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830456.522940:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477081216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830456.522950:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830456.522953:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830456.522956:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830456.522962:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830456.522981:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830456.522998:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830456/real 1787830456] req@ffff9a7890b1da40 x1874671477081344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830456.523008:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477081344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830456.523017:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830456.523020:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830456.523023:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830457.546234:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830458.570518:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830459.593979:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830460.618055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830461.642044:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830461.642063:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830461.642070:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830461.642075:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830461.642086:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4829 00000100:00080000:0.0:1787830461.642096:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830461.642102:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830461.642153:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477081472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830461.642181:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830461.642185:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830461.642186:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830461.642191:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4829 00000100:00080000:0.0:1787830461.642194:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830461.642198:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830461.642207:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830461.642220:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477081600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830461.642229:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830461.642232:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830461.642233:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830461.642237:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4829 00000100:00080000:0.0:1787830461.642240:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830461.642242:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830461.642247:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830461.642256:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477081728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830461.642268:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830461.642271:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830461.642273:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830461.642275:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4829 00000100:00080000:0.0:1787830461.642278:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830461.642281:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830461.642285:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830461.642294:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477081856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830461.642303:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830461.642306:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830461.642307:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830461.642310:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4829 00000100:00080000:0.0:1787830461.642313:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830461.642316:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830461.642319:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830461.642326:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477081984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830461.642382:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477081472:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830461.642431:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830461.642474:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830461/real 1787830461] req@ffff9a7890a3c000 x1874671477081472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830461.642489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477081472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830461.642498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830461.642502:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830461.642506:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477081472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830461.642512:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830461.642533:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830461.642551:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830461/real 1787830461] req@ffff9a7890a3fc00 x1874671477081600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830461.642562:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477081600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830461.642569:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830461.642572:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830461.642574:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830461.642579:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830461.642596:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830461.642606:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830461/real 1787830461] req@ffff9a7890a3de00 x1874671477081728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830461.642616:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477081728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830461.642623:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830461.642626:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830461.642629:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830461.642634:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830461.642651:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830461.642661:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830461/real 1787830461] req@ffff9a7890a3f480 x1874671477081856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830461.642678:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477081856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830461.642686:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830461.642690:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830461.642693:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830461.642698:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830461.642714:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830461.642724:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830461/real 1787830461] req@ffff9a7890a3da40 x1874671477081984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830461.642734:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477081984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830461.642743:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830461.642745:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830461.642748:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477081984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830462.665982:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830463.690131:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830464.714030:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830465.737985:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830466.762069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830466.763113:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830466.763125:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830466.763131:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830466.763145:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4834 00000100:00080000:0.0:1787830466.763158:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830466.763166:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830466.763228:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477082112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830466.763258:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830466.763261:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830466.763263:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830466.763541:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4834 00000100:00080000:0.0:1787830466.763545:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830466.763549:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830466.763622:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830466.763640:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477082240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830466.763646:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477082112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830466.763656:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830466.763661:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830466.763662:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830466.763668:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4834 00000100:00080000:0.0:1787830466.763672:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830466.763676:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830466.763681:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830466.763694:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477082368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830466.763703:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830466.763708:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830466.763710:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830466.763715:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4834 00000100:00080000:0.0:1787830466.763719:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787830466.763722:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830466.763727:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830466.763733:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830466.763743:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477082496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830466.763750:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830466.763753:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830466.763755:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830466.763759:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4834 00000100:00080000:0.0:1787830466.763762:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830466.763767:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830466.763771:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830466.763779:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477082624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830466.763792:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830466/real 1787830466] req@ffff9a7890a3c3c0 x1874671477082112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830466.763812:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477082112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830466.763822:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830466.763826:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830466.763863:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477082112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830466.763889:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830466.763912:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830466.763925:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830466/real 1787830466] req@ffff9a7890a3da40 x1874671477082240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830466.763937:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477082240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830466.763945:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830466.763948:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830466.763951:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830466.763957:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830466.763980:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830466.763992:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830466/real 1787830466] req@ffff9a7890a3f480 x1874671477082368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830466.764003:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477082368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830466.764011:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830466.764014:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830466.764018:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830466.764024:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830466.764041:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830466.764051:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830466/real 1787830466] req@ffff9a7890a3de00 x1874671477082496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830466.764061:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477082496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830466.764069:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830466.764072:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830466.764085:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830466.764090:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830466.764108:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830466.764119:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830466/real 1787830466] req@ffff9a7890a3fc00 x1874671477082624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830466.764129:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477082624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830466.764138:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830466.764141:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830466.764143:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830467.786197:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830468.809977:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830469.834042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830470.858429:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830471.881970:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830471.881981:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830471.881991:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787830471.881991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830471.882003:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4839 00000100:00080000:2.0:1787830471.882014:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830471.882020:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830471.882069:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477082752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830471.882107:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830471.882111:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830471.882113:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830471.882117:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4839 00000100:00080000:2.0:1787830471.882120:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830471.882125:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830471.882134:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830471.882158:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477082880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830471.882168:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830471.882171:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830471.882172:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830471.882176:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4839 00000100:00080000:2.0:1787830471.882178:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830471.882182:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830471.882186:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830471.882195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477083008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830471.882202:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830471.882205:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830471.882207:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830471.882210:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4839 00000100:00080000:2.0:1787830471.882212:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830471.882215:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830471.882219:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830471.882225:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477083136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830471.882232:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830471.882235:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830471.882236:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830471.882239:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4839 00000100:00080000:2.0:1787830471.882241:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830471.882244:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830471.882247:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830471.882254:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477083264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830471.882350:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477082752:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830471.882417:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830471.882467:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830471/real 1787830471] req@ffff9a78841da1c0 x1874671477082752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830471.882496:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477082752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830471.882507:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830471.882510:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830471.882514:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477082752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830471.882521:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830471.882541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830471.882552:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830471/real 1787830471] req@ffff9a78841d9a40 x1874671477082880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830471.882563:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477082880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830471.882570:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830471.882573:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830471.882576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477082880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830471.882582:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830471.882597:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830471.882606:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830471/real 1787830471] req@ffff9a78841d8780 x1874671477083008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830471.882622:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477083008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830471.882629:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830471.882632:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830471.882635:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830471.882640:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830471.882657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830471.882667:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830471/real 1787830471] req@ffff9a78841d9e00 x1874671477083136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830471.882677:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477083136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830471.882686:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830471.882689:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830471.882692:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830471.882697:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830471.882713:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830471.882722:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830471/real 1787830471] req@ffff9a78841dbc00 x1874671477083264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830471.882730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477083264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830471.882743:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830471.882745:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830471.882748:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830472.906220:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830473.932248:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830474.954223:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830475.978320:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830477.002115:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830477.002435:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830477.002443:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830477.002448:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830477.002463:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4844 00000100:00080000:1.0:1787830477.002477:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830477.002485:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830477.002551:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477083392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830477.002597:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830477.002600:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830477.002602:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830477.002605:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4844 00000100:00080000:1.0:1787830477.002608:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830477.002612:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830477.002624:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830477.002636:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477083520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830477.002645:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830477.002649:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830477.002650:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830477.002655:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4844 00000100:00080000:1.0:1787830477.002659:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830477.002663:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830477.002668:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830477.002681:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477083648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830477.002682:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477083392:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830477.002699:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830477.002707:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830477.002709:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830477.002713:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4844 00000100:00080000:1.0:1787830477.002717:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830477.002721:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830477.002727:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830477.002740:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477083776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830477.002748:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830477.002752:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830477.002754:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830477.002758:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4844 00000100:00080000:1.0:1787830477.002762:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830477.002766:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787830477.002772:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787830477.002772:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830477.002787:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477083904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830477.002887:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830477/real 1787830477] req@ffff9a7890b1da40 x1874671477083392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830477.002921:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477083392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830477.002939:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830477.002943:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830477.002948:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477083392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830477.002956:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830477.002984:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830477.002997:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830477/real 1787830477] req@ffff9a7890b1d680 x1874671477083520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830477.003023:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477083520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830477.003031:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830477.003035:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830477.003037:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830477.003063:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830477.003093:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830477.003103:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830477/real 1787830477] req@ffff9a7890b1de00 x1874671477083648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830477.003113:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477083648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830477.003120:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830477.003122:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830477.003126:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830477.003131:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830477.003149:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830477.003159:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830477/real 1787830477] req@ffff9a7890b1e580 x1874671477083776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830477.003169:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477083776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830477.003178:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830477.003181:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830477.003184:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830477.003190:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830477.003207:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830477.003218:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830477/real 1787830477] req@ffff9a7890b1e940 x1874671477083904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830477.003229:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477083904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830477.003239:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830477.003242:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830477.003245:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477083904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830478.027187:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830479.050023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830480.074283:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830481.097949:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830482.121931:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830482.121943:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830482.121947:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830482.121957:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4849 00000100:00080000:1.0:1787830482.121978:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830482.121987:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830482.122022:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477084032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830482.122048:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830482.122051:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830482.122053:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830482.122055:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4849 00000100:00080000:1.0:1787830482.122058:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830482.122061:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830482.122068:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830482.122078:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477084160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830482.122084:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830482.122086:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830482.122087:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830482.122090:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4849 00000100:00080000:1.0:1787830482.122092:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830482.122093:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830482.122095:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830482.122101:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477084288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830482.122112:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830482.122115:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830482.122119:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830482.122123:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4849 00000100:00080000:1.0:1787830482.122126:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830482.122129:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830482.122134:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830482.122143:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477084416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830482.122151:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830482.122154:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830482.122155:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830482.122158:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4849 00000100:00080000:1.0:1787830482.122161:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830482.122164:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830482.122168:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830482.122177:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477084544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830482.122368:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830482.122442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477084032:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830482.122500:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830482.122561:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830482/real 1787830482] req@ffff9a7890b1f0c0 x1874671477084032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830482.122580:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477084032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830482.122591:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830482.122595:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830482.122599:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477084032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830482.122606:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830482.122627:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830482.122642:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830482/real 1787830482] req@ffff9a7890b1f840 x1874671477084160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830482.122654:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477084160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830482.122662:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830482.122665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830482.122668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830482.122674:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830482.122693:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830482.122707:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830482/real 1787830482] req@ffff9a7890b1c000 x1874671477084288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830482.122720:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477084288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830482.122732:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830482.122735:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830482.122738:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830482.122746:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830482.122764:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830482.122775:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830482/real 1787830482] req@ffff9a7890b1f480 x1874671477084416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830482.122785:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477084416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830482.122795:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830482.122798:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830482.122801:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830482.122809:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830482.122827:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830482.122866:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830482/real 1787830482] req@ffff9a7890b1fc00 x1874671477084544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830482.122878:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477084544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830482.122887:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830482.122890:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830482.122893:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830483.146435:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830484.170405:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830485.194123:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830486.217970:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830487.241988:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830487.242002:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830487.242008:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787830487.242018:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830487.242021:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4854 00000100:00080000:1.0:1787830487.242037:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830487.242047:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830487.242102:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477084672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830487.242133:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830487.242137:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830487.242139:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830487.242143:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4854 00000100:00080000:1.0:1787830487.242146:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830487.242151:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830487.242163:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830487.242175:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477084800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830487.242184:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830487.242187:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830487.242189:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830487.242193:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4854 00000100:00080000:1.0:1787830487.242196:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830487.242199:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830487.242205:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830487.242216:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477084928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830487.242226:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830487.242241:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830487.242243:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830487.242247:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4854 00000100:00080000:1.0:1787830487.242250:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830487.242254:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830487.242260:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830487.242270:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477085056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830487.242288:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830487.242291:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830487.242293:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830487.242296:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4854 00000100:00080000:1.0:1787830487.242299:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830487.242303:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830487.242307:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830487.242316:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477085184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830487.243509:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477084672:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830487.243579:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830487.243643:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830487/real 1787830487] req@ffff9a7890b1fc00 x1874671477084672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830487.243803:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477084672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830487.243814:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830487.243819:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830487.243822:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477084672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830487.243876:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830487.243914:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830487.243928:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830487/real 1787830487] req@ffff9a7890b1f480 x1874671477084800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830487.243952:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477084800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830487.243960:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830487.243964:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830487.243967:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830487.243975:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830487.243996:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830487.244009:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830487/real 1787830487] req@ffff9a7890b1c000 x1874671477084928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830487.244021:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477084928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830487.244029:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830487.244033:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830487.244036:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477084928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830487.244043:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830487.244064:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830487.244076:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830487/real 1787830487] req@ffff9a7890b1f840 x1874671477085056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830487.244088:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477085056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830487.244098:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830487.244101:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830487.244105:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830487.244111:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830487.244130:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830487.244141:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830487/real 1787830487] req@ffff9a7890b1f0c0 x1874671477085184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830487.244153:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477085184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830487.244162:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830487.244165:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830487.244169:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830488.267088:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830489.290072:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830490.313960:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830491.338163:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830492.362061:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830492.362070:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830492.362075:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830492.362083:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4859 00000100:00080000:1.0:1787830492.362093:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830492.362098:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830492.362130:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477085312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830492.362162:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830492.362165:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830492.362166:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830492.362169:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4859 00000100:00080000:1.0:1787830492.362172:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830492.362176:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830492.362185:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830492.362195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477085440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830492.362203:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830492.362205:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830492.362207:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830492.362211:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4859 00000100:00080000:1.0:1787830492.362213:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830492.362216:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830492.362219:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830492.362227:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477085568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830492.362240:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830492.362242:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830492.362244:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830492.362247:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4859 00000100:00080000:1.0:1787830492.362249:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830492.362251:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830492.362254:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830492.362261:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477085696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830492.362267:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830492.362269:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830492.362270:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830492.362273:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4859 00000100:00080000:1.0:1787830492.362276:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830492.362278:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830492.362281:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830492.362288:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477085824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830492.362448:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477085312:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830492.362506:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830492.362716:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830492/real 1787830492] req@ffff9a7890b1e940 x1874671477085312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787830492.362722:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830492.362734:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477085312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830492.362746:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830492.362750:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830492.362754:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477085312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830492.362760:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830492.362782:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830492.362794:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830492/real 1787830492] req@ffff9a7890b1e580 x1874671477085440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830492.362805:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477085440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830492.362811:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830492.362813:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830492.362815:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830492.362819:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830492.362852:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830492.362860:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830492/real 1787830492] req@ffff9a7890b1de00 x1874671477085568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830492.362866:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477085568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830492.362871:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830492.362873:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830492.362875:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830492.362878:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830492.362889:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830492.362894:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830492/real 1787830492] req@ffff9a7890b1d680 x1874671477085696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830492.362915:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477085696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830492.362923:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830492.362926:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830492.362929:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830492.362950:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830492.362963:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830492.362970:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830492/real 1787830492] req@ffff9a7890b1da40 x1874671477085824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830492.362977:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477085824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830492.362983:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830492.362985:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830492.362986:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477085824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830493.386023:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830494.410104:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830495.434055:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830496.458081:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830497.481949:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830497.481959:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830497.481963:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830497.481973:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4864 00000100:00080000:0.0:1787830497.481982:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830497.481990:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830497.482024:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477085952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830497.482053:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830497.482057:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830497.482059:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830497.482062:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4864 00000100:00080000:0.0:1787830497.482066:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830497.482070:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830497.482078:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830497.482092:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477086080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830497.482108:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830497.482111:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830497.482112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830497.482116:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4864 00000100:00080000:0.0:1787830497.482119:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830497.482122:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830497.482126:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830497.482136:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477086208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830497.482144:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830497.482147:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830497.482149:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830497.482153:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4864 00000100:00080000:0.0:1787830497.482156:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830497.482160:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830497.482164:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830497.482172:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477086336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830497.482180:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830497.482183:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830497.482184:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830497.482188:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4864 00000100:00080000:0.0:1787830497.482191:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830497.482193:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830497.482197:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830497.482204:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477086464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830497.482343:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477085952:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830497.482401:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830497.482451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830497/real 1787830497] req@ffff9a7890a3c000 x1874671477085952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830497.482474:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477085952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830497.482483:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830497.482487:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830497.482490:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477085952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830497.482496:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830497.482517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830497.482529:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830497/real 1787830497] req@ffff9a7890a3fc00 x1874671477086080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830497.482540:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477086080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830497.482548:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830497.482551:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830497.482554:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830497.482560:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830497.482579:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830497.482591:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830497/real 1787830497] req@ffff9a7890a3de00 x1874671477086208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830497.482601:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477086208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830497.482614:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830497.482617:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830497.482620:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086208:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787830497.482713:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830497.482744:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830497.482769:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830497.482781:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830497/real 1787830497] req@ffff9a7890a3f480 x1874671477086336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830497.482793:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477086336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830497.482801:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830497.482804:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830497.482807:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830497.482812:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830497.482850:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830497.482871:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830497/real 1787830497] req@ffff9a7890a3da40 x1874671477086464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830497.482883:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477086464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830497.482891:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830497.482893:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830497.482897:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830498.506002:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830499.529975:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830500.554421:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830501.578880:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830502.602352:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830502.602363:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830502.602368:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830502.602377:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4870 00000100:00080000:2.0:1787830502.602388:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830502.602396:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830502.602434:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477086592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830502.602506:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477086592:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830502.602567:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830502.602598:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830502.602602:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830502.602604:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830502.602608:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4870 00000100:00080000:2.0:1787830502.602613:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830502.602617:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830502.602627:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830502.602640:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477086720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830502.604247:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830502/real 1787830502] req@ffff9a78841dbc00 x1874671477086592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830502.604273:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477086592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830502.604284:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830502.604289:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830502.604293:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477086592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830502.604323:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830502.604365:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830502.604382:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830502/real 1787830502] req@ffff9a78841d9e00 x1874671477086720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830502.604393:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477086720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830502.604403:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830502.604407:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830502.604410:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086720:192.168.203.140@tcp:38:kworker.0 00010000:00010000:3.0:1787830502.604481:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830502.608322:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830502.608331:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830502.608334:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830502.608343:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4870 00000100:00080000:2.0:1787830502.608353:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830502.608360:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830502.608373:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830502.608407:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477086848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830502.608422:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830502.608425:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830502.608427:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830502.608430:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4870 00000100:00080000:2.0:1787830502.608434:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830502.608437:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830502.608441:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830502.608451:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477086976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830502.608459:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830502.608462:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830502.608464:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830502.608468:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4870 00000100:00080000:2.0:1787830502.608471:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830502.608475:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830502.608478:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830502.608490:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477087104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830502.609285:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830502.609342:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830502.609369:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830502/real 1787830502] req@ffff9a78841d8780 x1874671477086848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830502.609384:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477086848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830502.609393:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830502.609396:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830502.609399:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830502.609405:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830502.609421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830502.609430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830502/real 1787830502] req@ffff9a78841d9a40 x1874671477086976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830502.609439:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477086976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830502.609447:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830502.609450:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830502.609452:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477086976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830502.609458:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830502.609472:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830502.609481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830502/real 1787830502] req@ffff9a78841da1c0 x1874671477087104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830502.609491:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477087104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830502.609499:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830502.609502:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830502.609505:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830503.626286:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830504.650039:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830505.673971:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830506.698371:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830507.722027:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830507.722035:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830507.722039:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830507.722047:0:36:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4875 00010000:00010000:1.0:1787830507.722050:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830507.722057:0:36:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830507.722072:0:36:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830507.722110:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477087232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830507.722131:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830507.722135:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830507.722136:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830507.722140:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4875 00000100:00080000:2.0:1787830507.722144:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830507.722147:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830507.722157:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830507.722168:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477087360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830507.722176:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830507.722178:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830507.722179:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830507.722182:0:36:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4875 00000100:00080000:2.0:1787830507.722184:0:36:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830507.722186:0:36:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830507.722188:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830507.722195:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477087488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830507.722200:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830507.722202:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830507.722203:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830507.722205:0:36:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4875 00000100:00080000:2.0:1787830507.722206:0:36:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830507.722208:0:36:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830507.722211:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830507.722215:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477087616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830507.722219:0:36:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830507.722221:0:36:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830507.722222:0:36:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830507.722224:0:36:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4875 00000100:00080000:2.0:1787830507.722226:0:36:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830507.722228:0:36:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830507.722230:0:36:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830507.722235:0:36:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477087744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830507.722269:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477087232:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830507.722328:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830507.722373:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830507/real 1787830507] req@ffff9a78841db0c0 x1874671477087232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830507.722390:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477087232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830507.722400:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830507.722404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830507.722408:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477087232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830507.722415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830507.722438:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830507.722451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830507/real 1787830507] req@ffff9a78841d83c0 x1874671477087360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830507.722463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477087360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830507.722471:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830507.722475:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830507.722479:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830507.722485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830507.722505:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830507.722516:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830507/real 1787830507] req@ffff9a78841db480 x1874671477087488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830507.722529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477087488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830507.722537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830507.722540:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830507.722544:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830507.722549:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830507.722569:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830507.722580:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830507/real 1787830507] req@ffff9a78841da580 x1874671477087616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830507.722592:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477087616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830507.722602:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830507.722605:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830507.722609:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830507.722615:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830507.722633:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830507.722644:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830507/real 1787830507] req@ffff9a78841d8000 x1874671477087744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830507.722655:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477087744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830507.722665:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830507.722668:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830507.722671:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477087744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830508.746178:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830509.771104:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830510.794821:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830511.817968:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830512.841993:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830512.842003:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830512.842007:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830512.842016:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4880 00000100:00080000:2.0:1787830512.842026:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830512.842032:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830512.842069:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477087872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830512.842148:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477087872:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830512.842206:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830512.842235:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830512.842240:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830512.842242:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830512.842246:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4880 00000100:00080000:2.0:1787830512.842250:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830512.842255:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830512.842264:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830512.842278:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477088000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830512.843090:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830512/real 1787830512] req@ffff9a78841d92c0 x1874671477087872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830512.843111:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477087872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830512.843122:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830512.843125:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830512.843129:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477087872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830512.843150:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830512.843249:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830512.843265:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830512/real 1787830512] req@ffff9a78841db840 x1874671477088000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830512.843277:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477088000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830512.843284:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830512.843287:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830512.843291:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088000:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787830512.843774:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830512.845438:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830512.845446:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830512.845450:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830512.845458:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4880 00000100:00080000:2.0:1787830512.845467:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830512.845474:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830512.845486:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830512.845527:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477088128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830512.845540:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830512.845543:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830512.845545:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787830512.845661:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830512.845714:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830512.846040:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4880 00000100:00080000:2.0:1787830512.846045:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830512.846050:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830512.846057:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830512.846071:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477088256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830512.846082:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830512.846085:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830512.846087:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830512.846091:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4880 00000100:00080000:2.0:1787830512.846094:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830512.846098:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830512.846103:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830512.846113:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477088384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830512.846249:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830512/real 1787830512] req@ffff9a78841d9680 x1874671477088128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830512.846277:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477088128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830512.846286:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830512.846298:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830512.846302:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830512.846321:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830512.846358:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830512.846370:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830512/real 1787830512] req@ffff9a78841d8b40 x1874671477088256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830512.846381:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477088256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830512.846391:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830512.846394:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830512.846397:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830512.846403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830512.846420:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830512.846431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830512/real 1787830512] req@ffff9a78841da940 x1874671477088384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830512.846442:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477088384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830512.846451:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830512.846453:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830512.846456:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830513.866188:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830514.890226:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830515.914310:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830516.939244:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830517.962261:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830517.962270:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830517.962274:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830517.962282:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4885 00000100:00080000:3.0:1787830517.962290:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830517.962295:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830517.962325:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477088512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830517.962342:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830517.962345:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830517.962346:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830517.962348:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4885 00000100:00080000:3.0:1787830517.962350:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830517.962352:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830517.962359:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830517.962366:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477088640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830517.962372:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830517.962375:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830517.962378:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830517.962381:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4885 00000100:00080000:3.0:1787830517.962383:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830517.962385:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830517.962387:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830517.962394:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477088768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830517.962401:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830517.962403:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830517.962405:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830517.962413:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4885 00000100:00080000:3.0:1787830517.962415:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830517.962417:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477088512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830517.962419:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830517.962429:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830517.962444:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671477088896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830517.962449:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830517.962451:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830517.962452:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830517.962455:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4885 00000100:00080000:3.0:1787830517.962464:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830517.962467:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830517.962471:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830517.962483:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671477089024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787830517.962536:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830517.962576:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830517.962776:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830517.962807:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830517.962867:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830518.986270:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830520.010316:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830521.033977:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787830521.034096:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787830521.034134:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787830521.034149:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787830521.037162:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830517/real 1787830521] req@ffff9a78a6776d00 x1874671477088512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830542 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830521.037208:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477088512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830542 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830521.037225:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830521.037230:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830521.037235:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477088512:192.168.203.140@tcp:250:kworker.0 00000800:00000100:2.0:1787830521.038399:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830521.038428:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830521.038437:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830521.038444:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787830521.038523:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830517/real 1787830521] req@ffff9a78a6774780 x1874671477088768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830521.038546:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477088768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830521.038556:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830521.038561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830521.038565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830521.038572:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830517/real 1787830521] req@ffff9a78a6775680 x1874671477088896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830521.038585:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671477088896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830521.038595:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830521.038598:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830521.038602:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830521.038608:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830517/real 1787830521] req@ffff9a78a67770c0 x1874671477089024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830521.038620:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671477089024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830521.038629:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830521.038633:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830521.038636:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089024:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830521.038643:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830517/real 1787830521] req@ffff9a78a67761c0 x1874671477088640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830521.038655:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477088640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830542 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830521.038663:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830521.038666:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830521.038669:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477088640:192.168.203.140@tcp:38:kworker.0 00010000:00010000:1.0:1787830522.058061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830523.081958:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830523.081969:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830523.081973:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830523.081982:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4890 00000100:00080000:3.0:1787830523.081993:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830523.082000:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830523.082037:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477089152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830523.082067:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830523.082071:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830523.082073:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830523.082077:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4890 00000100:00080000:3.0:1787830523.082080:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830523.082084:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830523.082093:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830523.082105:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477089280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830523.082115:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830523.082118:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830523.082119:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830523.082123:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4890 00000100:00080000:3.0:1787830523.082126:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830523.082130:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830523.082133:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830523.082144:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477089408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830523.082153:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830523.082156:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830523.082158:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830523.082161:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4890 00000100:00080000:3.0:1787830523.082164:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830523.082168:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830523.082172:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830523.082181:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477089536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830523.082189:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830523.082192:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830523.082193:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830523.082196:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4890 00000100:00080000:3.0:1787830523.082199:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830523.082202:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830523.082205:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830523.082213:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477089664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830523.082331:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830523.082523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477089152:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830523.082577:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830523.082617:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830523/real 1787830523] req@ffff9a78a6776940 x1874671477089152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830523.082634:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477089152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830523.082643:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830523.082647:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830523.082651:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477089152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830523.082658:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830523.082680:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830523.082704:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830523/real 1787830523] req@ffff9a78a6776580 x1874671477089280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830523.082715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477089280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830523.082727:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830523.082730:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830523.082733:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830523.082738:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830523.082756:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830523.082766:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830523/real 1787830523] req@ffff9a78a6774b40 x1874671477089408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830523.082775:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477089408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830523.082783:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830523.082786:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830523.082789:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830523.082794:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830523.082814:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830523.082824:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830523/real 1787830523] req@ffff9a78a67752c0 x1874671477089536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830523.082866:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477089536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830523.082876:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830523.082879:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830523.082882:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830523.082888:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830523.082905:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830523.082915:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830523/real 1787830523] req@ffff9a78a6775a40 x1874671477089664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830523.082925:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477089664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830523.082933:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830523.082935:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830523.082938:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830524.105988:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830525.130105:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830526.154051:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830527.178113:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830528.202039:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830528.202059:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830528.202064:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830528.202074:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4895 00000100:00080000:3.0:1787830528.202084:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830528.202091:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830528.202131:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671477089792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830528.202160:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830528.202164:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830528.202166:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830528.202169:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4895 00000100:00080000:3.0:1787830528.202173:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830528.202177:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830528.202187:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830528.202209:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477089792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830528.202211:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671477089920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830528.202225:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830528.202228:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830528.202230:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830528.202234:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4895 00000100:00080000:3.0:1787830528.202239:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830528.202243:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830528.202248:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830528.202261:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477090048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830528.202271:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1787830528.202272:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787830528.202276:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830528.202278:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830528.202282:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4895 00000100:00080000:3.0:1787830528.202285:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830528.202290:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830528.202295:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830528.202308:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477090176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830528.202317:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830528.202320:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830528.202322:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830528.202327:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4895 00000100:00080000:3.0:1787830528.202330:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830528.202336:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830528.202340:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830528.202347:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830528/real 1787830528] req@ffff9a78a6777480 x1874671477089792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830528.202352:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477090304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830528.202389:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671477089792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830528.202400:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830528.202404:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830528.202408:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477089792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830528.202431:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830528.202455:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830528.202468:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830528/real 1787830528] req@ffff9a78a6774000 x1874671477089920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830528.202480:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671477089920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830528.202488:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830528.202491:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830528.202494:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477089920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830528.202499:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830528.202517:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830528.202527:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830528/real 1787830528] req@ffff9a78a6775a40 x1874671477090048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830528.202538:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477090048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830528.202546:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830528.202549:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830528.202551:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830528.202557:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830528.202576:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830528.202586:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830528/real 1787830528] req@ffff9a78a67752c0 x1874671477090176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830528.202595:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477090176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830528.202604:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830528.202607:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830528.202610:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830528.202620:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830528.202660:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:1.0:1787830528.202661:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830528.202675:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830528/real 1787830528] req@ffff9a78a6774b40 x1874671477090304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830528.202687:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477090304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830528.202695:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830528.202697:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830528.202700:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830529.226237:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830530.249976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830531.274039:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830532.298078:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830533.322008:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830533.322078:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830533.322084:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830533.322088:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830533.322107:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4900 00000100:00080000:0.0:1787830533.322117:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830533.322123:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830533.322157:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477090432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830533.322186:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830533.322190:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830533.322192:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830533.322195:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4900 00000100:00080000:0.0:1787830533.322199:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830533.322202:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830533.322210:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830533.322223:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477090560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830533.322233:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830533.322236:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830533.322238:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830533.322241:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4900 00000100:00080000:0.0:1787830533.322245:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830533.322248:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830533.322252:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830533.322266:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477090688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830533.322274:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830533.322277:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830533.322279:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830533.322283:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4900 00000100:00080000:0.0:1787830533.322286:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830533.322289:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830533.322294:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830533.322302:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477090816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830533.322311:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830533.322313:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830533.322315:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830533.322318:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4900 00000100:00080000:0.0:1787830533.322321:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830533.322337:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830533.322341:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830533.322349:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477090944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830533.322670:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477090432:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830533.322729:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830533.322801:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830533/real 1787830533] req@ffff9a7890a3c3c0 x1874671477090432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830533.322818:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477090432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830533.322828:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830533.322858:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830533.322862:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477090432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830533.322869:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830533.322892:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830533.322905:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830533/real 1787830533] req@ffff9a7890a3da40 x1874671477090560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830533.322917:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477090560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830533.322924:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830533.322927:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830533.322930:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830533.322936:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830533.322956:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830533.322969:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830533/real 1787830533] req@ffff9a7890a3f480 x1874671477090688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830533.322981:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477090688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830533.322988:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830533.322992:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830533.322995:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830533.323001:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830533.323021:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830533.323032:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830533/real 1787830533] req@ffff9a7890a3de00 x1874671477090816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830533.323042:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477090816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830533.323051:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830533.323054:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830533.323057:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830533.323063:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830533.323081:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830533.323092:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830533/real 1787830533] req@ffff9a7890a3fc00 x1874671477090944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830533.323103:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477090944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830533.323112:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830533.323115:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830533.323118:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477090944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830534.346055:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830535.370081:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830536.394466:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830537.418316:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830538.442065:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830538.442075:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830538.442079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830538.442089:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4905 00000100:00080000:3.0:1787830538.442100:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830538.442107:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830538.442142:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477091072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830538.442169:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830538.442172:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830538.442174:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830538.442178:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4905 00000100:00080000:3.0:1787830538.442182:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830538.442185:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830538.442193:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830538.442204:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477091200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830538.442212:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830538.442214:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830538.442216:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830538.442220:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4905 00000100:00080000:3.0:1787830538.442223:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830538.442226:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830538.442230:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830538.442240:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477091328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830538.442249:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830538.442252:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830538.442254:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830538.442258:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4905 00000100:00080000:3.0:1787830538.442262:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830538.442263:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477091072:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830538.442471:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830538.442760:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:0.0:1787830538.442763:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:3.0:1787830538.442777:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830538.442793:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671477091456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830538.442806:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830538.442809:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830538.442813:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787830538.442814:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830538/real 1787830538] req@ffff9a78a6776580 x1874671477091072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830538.442821:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4905 00000100:00100000:0.0:1787830538.442849:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477091072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830538.442857:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830538.442859:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830538.442861:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830538.442868:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477091072:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830538.442868:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830538.442878:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830538.442879:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830538.442896:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671477091584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830538.442906:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830538.442919:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830538/real 1787830538] req@ffff9a78a6776940 x1874671477091200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830538.442927:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477091200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830538.442932:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830538.442935:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830538.442936:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830538.442940:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830538.443320:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830538.443334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830538/real 1787830538] req@ffff9a78a67761c0 x1874671477091328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830538.443345:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477091328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830538.443353:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830538.443361:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830538.443364:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830538.443388:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830538.443405:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830538.443413:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830538/real 1787830538] req@ffff9a78a67770c0 x1874671477091456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830538.443424:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671477091456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830538.443430:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830538.443432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830538.443434:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830538.443437:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830538.443446:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830538.443451:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830538/real 1787830538] req@ffff9a78a6775680 x1874671477091584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830538.443457:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671477091584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830538.443463:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830538.443464:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830538.443466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:3.0:1787830539.465967:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830540.491251:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830541.514718:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830542.538046:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830543.562125:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830543.562523:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830543.562529:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830543.562533:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830543.562613:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4911 00000100:00080000:2.0:1787830543.562623:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830543.562630:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830543.562668:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477091712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830543.562696:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830543.562699:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830543.562701:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830543.562704:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4911 00000100:00080000:2.0:1787830543.562707:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830543.562711:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830543.562720:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830543.562731:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477091840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830543.562740:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830543.562742:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830543.562744:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830543.562747:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4911 00000100:00080000:2.0:1787830543.562750:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830543.562753:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830543.562757:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830543.562815:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477091968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830543.562826:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830543.562855:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830543.562858:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830543.562863:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4911 00000100:00080000:2.0:1787830543.562867:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830543.562879:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477091712:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830543.562948:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830543.562992:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830543/real 1787830543] req@ffff9a78841da940 x1874671477091712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830543.563010:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830543.563011:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477091712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787830543.563020:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:0.0:1787830543.563035:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830543.563040:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830543.563043:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477091712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830543.563047:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477092096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830543.563053:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830543.563076:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830543.563087:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830543/real 1787830543] req@ffff9a78841d8b40 x1874671477091840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830543.563098:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477091840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830543.563106:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830543.563109:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830543.563112:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830543.563118:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830543.563133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830543.563142:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830543/real 1787830543] req@ffff9a78841d9680 x1874671477091968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830543.563153:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477091968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830543.563176:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830543.563180:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830543.563181:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830543.563188:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477091968:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830543.563189:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830543.563194:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830543.563201:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4911 00000100:00080000:2.0:1787830543.563205:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830543.563208:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830543.563212:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830543.563216:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830543.563225:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477092224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830543.563237:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830543.563250:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830543/real 1787830543] req@ffff9a78841db840 x1874671477092096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830543.563261:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477092096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830543.563271:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830543.563274:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830543.563277:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830543.563286:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830543.563299:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830543.563307:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830543/real 1787830543] req@ffff9a78841d92c0 x1874671477092224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830543.563318:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477092224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830543.563327:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830543.563330:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830543.563333:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830544.587779:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830545.609984:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830546.634110:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830547.658211:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830548.681951:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830548.682337:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830548.682343:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830548.682347:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830548.682357:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4916 00000100:00080000:3.0:1787830548.682366:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830548.682556:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830548.682594:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477092352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830548.682622:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830548.682626:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830548.682627:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830548.682631:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4916 00000100:00080000:3.0:1787830548.682636:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830548.682639:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830548.682648:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830548.682659:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477092480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830548.682668:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830548.682671:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830548.682673:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830548.682676:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4916 00000100:00080000:3.0:1787830548.682679:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830548.682682:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830548.682685:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830548.682696:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671477092608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830548.682703:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830548.682706:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830548.682707:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830548.682710:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4916 00000100:00080000:3.0:1787830548.682713:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830548.682716:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830548.682720:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830548.682728:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671477092736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830548.682737:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830548.682740:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830548.682741:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830548.682744:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4916 00000100:00080000:3.0:1787830548.682747:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830548.682750:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830548.682754:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830548.682763:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671477092864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830548.683063:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477092352:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830548.683122:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830548.683169:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830548/real 1787830548] req@ffff9a78a6774780 x1874671477092352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830548.683185:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477092352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830548.683193:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830548.683197:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830548.683200:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477092352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830548.683207:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830548.683228:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830548.683240:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830548/real 1787830548] req@ffff9a78a6776d00 x1874671477092480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830548.683251:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477092480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830548.683259:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830548.683262:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830548.683265:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830548.683271:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830548.683289:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830548.683299:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830548/real 1787830548] req@ffff9a78a6775e00 x1874671477092608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830548.683311:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671477092608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830548.683323:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830548.683326:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830548.683329:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830548.683335:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830548.683353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830548.683364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830548/real 1787830548] req@ffff9a78a6777c00 x1874671477092736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830548.683375:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671477092736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830548.683384:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830548.683387:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830548.683390:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830548.683396:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830548.683414:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830548.683423:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830548/real 1787830548] req@ffff9a78a6777840 x1874671477092864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830548.683432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671477092864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830548.683440:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830548.683443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830548.683446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477092864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830549.706069:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830550.730204:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830551.754031:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830552.778018:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830553.802582:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830553.802591:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830553.802595:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830553.802604:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4921 00000100:00080000:3.0:1787830553.802625:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830553.802632:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830553.802668:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671477092992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830553.802700:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830553.802704:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830553.802706:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830553.802709:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4921 00000100:00080000:3.0:1787830553.802713:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830553.802717:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830553.802726:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830553.802738:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671477093120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830553.802748:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830553.802751:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830553.802753:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830553.802756:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4921 00000100:00080000:3.0:1787830553.802759:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830553.802762:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830553.802766:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830553.802777:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671477093248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830553.802786:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830553.802789:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830553.802791:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830553.802794:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4921 00000100:00080000:3.0:1787830553.802797:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830553.802800:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830553.802804:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830553.802815:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671477093376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830553.802823:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830553.802826:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830553.802828:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830553.802860:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4921 00000100:00080000:3.0:1787830553.802864:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830553.802867:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830553.802872:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830553.802883:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671477093504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830553.803198:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477092992:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830553.803329:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830553.803383:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830553/real 1787830553] req@ffff9a78a67743c0 x1874671477092992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830553.803400:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671477092992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830553.803409:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830553.803413:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830553.803416:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477092992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830553.803424:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830553.803447:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830553.803459:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830553/real 1787830553] req@ffff9a78a6774f00 x1874671477093120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830553.803478:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671477093120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830553.803486:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830553.803489:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830553.803492:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830553.803498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830553.803516:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830553.803533:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830553/real 1787830553] req@ffff9a78a6777840 x1874671477093248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830553.803544:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671477093248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830553.803553:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830553.803556:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830553.803559:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830553.803564:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830553.803584:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830553.803595:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830553/real 1787830553] req@ffff9a78a6777c00 x1874671477093376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830553.803607:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671477093376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830553.804012:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830553.804019:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830553.804023:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830553.804030:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830553.804066:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830553.804079:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830553/real 1787830553] req@ffff9a78a6775e00 x1874671477093504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830553.804096:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671477093504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830553.804105:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830553.804108:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830553.804111:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830553.806427:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830554.827069:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830555.850899:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830556.875142:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830557.898016:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830558.922401:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830558.922416:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830558.922422:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830558.922443:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4926 00000100:00080000:3.0:1787830558.922458:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830558.922467:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830558.922518:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477093632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830558.922558:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830558.922563:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830558.922565:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830558.922569:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4926 00000100:00080000:3.0:1787830558.922572:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830558.922576:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830558.922588:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830558.922600:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477093760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830558.922610:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830558.922613:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830558.922615:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830558.922619:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4926 00000100:00080000:3.0:1787830558.922622:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830558.922626:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830558.922630:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830558.922642:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671477093888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830558.922650:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830558.922654:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830558.922655:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830558.922659:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4926 00000100:00080000:3.0:1787830558.922662:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830558.922665:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830558.922670:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830558.922679:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671477094016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830558.922687:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830558.922690:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830558.922692:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830558.922696:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4926 00000100:00080000:3.0:1787830558.922699:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830558.922702:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830558.922706:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830558.922715:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477094144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830558.922994:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477093632:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830558.923072:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830558.923129:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830558/real 1787830558] req@ffff9a78a6776d00 x1874671477093632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830558.923148:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477093632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830558.923159:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830558.923164:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830558.923167:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477093632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830558.923175:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830558.923198:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830558.923210:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830558/real 1787830558] req@ffff9a78a6774780 x1874671477093760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830558.923221:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477093760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830558.923229:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830558.923232:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830558.923236:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830558.923242:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830558.923259:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830558.923268:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830558/real 1787830558] req@ffff9a78a6775680 x1874671477093888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830558.923279:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671477093888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830558.923287:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830558.923290:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830558.923293:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477093888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830558.923298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830558.923316:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830558.923326:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830558/real 1787830558] req@ffff9a78a67770c0 x1874671477094016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830558.923337:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671477094016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830558.923346:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830558.923350:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830558.923353:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830558.923358:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830558.923376:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830558.923386:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830558/real 1787830558] req@ffff9a78a67761c0 x1874671477094144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830558.923397:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477094144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830558.923406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830558.923409:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830558.923412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830558.923502:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830559.946016:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830560.969976:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830561.994228:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830563.018039:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830564.041973:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830564.041982:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830564.041987:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830564.041997:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4931 00000100:00080000:0.0:1787830564.042008:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830564.042014:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:3.0:1787830564.042022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830564.042050:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477094272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830564.042081:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830564.042085:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830564.042087:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830564.042091:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4931 00000100:00080000:0.0:1787830564.042094:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830564.042104:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830564.042112:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830564.042122:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477094400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830564.042132:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830564.042134:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830564.042136:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830564.042138:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4931 00000100:00080000:0.0:1787830564.042140:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830564.042142:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830564.042148:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830564.042156:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477094528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830564.042163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830564.042166:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830564.042167:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830564.042169:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4931 00000100:00080000:0.0:1787830564.042171:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830564.042173:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830564.042175:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830564.042182:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477094656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830564.042187:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830564.042189:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830564.042190:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830564.042192:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4931 00000100:00080000:0.0:1787830564.042195:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830564.042197:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830564.042199:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830564.042203:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477094784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830564.042297:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477094272:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830564.042353:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830564.042392:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830564/real 1787830564] req@ffff9a7890a3c000 x1874671477094272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830564.042417:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477094272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830564.042428:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830564.042432:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830564.042435:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477094272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830564.042442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830564.042468:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830564.042481:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830564/real 1787830564] req@ffff9a7890a3fc00 x1874671477094400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830564.042493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477094400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830564.042500:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830564.042502:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830564.042504:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830564.042508:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830564.042521:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830564.042528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830564/real 1787830564] req@ffff9a7890a3de00 x1874671477094528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830564.042535:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477094528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830564.042539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830564.042541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830564.042543:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830564.042546:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830564.042554:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830564.042559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830564/real 1787830564] req@ffff9a7890a3f480 x1874671477094656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830564.042570:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477094656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830564.042575:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830564.042577:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830564.042578:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830564.042581:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830564.042589:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830564.042593:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830564/real 1787830564] req@ffff9a7890a3da40 x1874671477094784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830564.042603:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477094784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830564.042612:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830564.042615:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830564.042617:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477094784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830565.067120:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830566.090414:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830567.114149:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830568.138090:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830569.161947:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830569.161958:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830569.161962:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830569.161990:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4936 00000100:00080000:3.0:1787830569.162000:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830569.162008:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830569.162043:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477094912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830569.162085:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830569.162089:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830569.162092:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830569.162096:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4936 00000100:00080000:3.0:1787830569.162100:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830569.162107:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830569.162108:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477094912:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787830569.162117:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830569.162133:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477095040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830569.162143:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830569.162146:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830569.162148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830569.162153:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4936 00000800:00000100:0.0:1787830569.162208:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787830569.162236:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830569.162243:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830569.162249:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830569.162264:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477095168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830569.162282:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830569.162285:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830569.162287:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830569.162293:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4936 00000100:00100000:0.0:1787830569.162294:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830569/real 1787830569] req@ffff9a78a6776940 x1874671477094912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830569.162297:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830569.162304:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830569.162313:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830569.162322:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477094912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830569.162332:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477095296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830569.162333:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830569.162343:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:3.0:1787830569.162350:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787830569.162351:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477094912:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830569.162357:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830569.162360:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830569.162375:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4936 00000100:00080000:3.0:1787830569.162379:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830569.162382:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830569.162386:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830569.162393:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830569.162399:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477095424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830569.162424:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830569.162439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830569/real 1787830569] req@ffff9a78a6776580 x1874671477095040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830569.162452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477095040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830569.162460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830569.162464:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830569.162469:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830569.162475:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830569.162495:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830569.162506:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830569/real 1787830569] req@ffff9a78a6774b40 x1874671477095168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830569.162530:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477095168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830569.162538:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830569.162541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830569.162544:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830569.162549:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830569.162569:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830569.162580:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830569/real 1787830569] req@ffff9a78a67752c0 x1874671477095296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830569.162591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477095296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830569.162600:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830569.162603:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830569.162605:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830569.162621:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830569.162637:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830569.162646:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830569/real 1787830569] req@ffff9a78a6775a40 x1874671477095424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830569.162656:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477095424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830569.162665:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830569.162667:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830569.162670:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830569.162767:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830570.185949:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830571.210189:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830572.236376:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830573.257934:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830574.282106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830574.282115:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830574.282119:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830574.282129:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4941 00000100:00080000:3.0:1787830574.282140:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830574.282146:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787830574.282153:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830574.282181:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671477095552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830574.282204:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830574.282208:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830574.282210:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830574.282213:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4941 00000100:00080000:3.0:1787830574.282216:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830574.282219:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830574.282266:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477095552:192.168.203.140@tcp:250:kworker.0 00000008:00100000:3.0:1787830574.282301:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830574.282318:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671477095680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787830574.282340:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787830574.282344:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830574.282348:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830574.282349:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830574.282355:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4941 00000100:00080000:3.0:1787830574.282363:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830574.282367:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830574.282371:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830574.282386:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477095808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830574.282394:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830574/real 1787830574] req@ffff9a78a6774000 x1874671477095552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830574.282397:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830574.282405:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830574.282408:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830574.282417:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4941 00000100:00080000:3.0:1787830574.282422:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830574.282427:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671477095552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830574.282428:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830574.282435:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00080000:0.0:1787830574.282446:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1787830574.282451:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477095936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830574.282452:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830574.282461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477095552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830574.282468:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830574.282473:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830574.282475:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830574.282485:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4941 00000100:00080000:3.0:1787830574.282488:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830574.282491:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095680:192.168.203.140@tcp:38:kworker.0 00000100:00080000:3.0:1787830574.282499:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830574.282507:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:0.0:1787830574.282519:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830574.282525:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477096064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830574.282537:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830574/real 1787830574] req@ffff9a78a6777480 x1874671477095680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830574.282559:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671477095680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830574.282575:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830574.282578:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830574.282582:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830574.282588:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830574.282607:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830574.282617:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830574/real 1787830574] req@ffff9a78a6775a40 x1874671477095808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830574.282628:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477095808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830574.282636:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830574.282639:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830574.282660:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830574.282666:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830574.282690:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830574.282701:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830574/real 1787830574] req@ffff9a78a67752c0 x1874671477095936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830574.282712:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477095936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830574.282721:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830574.282724:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830574.282728:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477095936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830574.282743:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830574.282761:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830574.282772:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830574/real 1787830574] req@ffff9a78a6774b40 x1874671477096064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830574.282784:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477096064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830574.282793:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830574.282796:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830574.282799:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830575.306403:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787830576.330434:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830577.354152:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830578.378968:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830579.401993:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830579.402003:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830579.402007:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830579.402016:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4946 00000100:00080000:3.0:1787830579.402027:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830579.402033:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830579.402067:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477096192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830579.402098:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830579.402103:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830579.402104:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830579.402108:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4946 00000100:00080000:3.0:1787830579.402111:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830579.402116:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830579.402124:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830579.402135:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477096320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830579.402144:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830579.402147:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830579.402149:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830579.402153:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4946 00000100:00080000:3.0:1787830579.402156:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830579.402159:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830579.402163:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830579.402173:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477096448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830579.402182:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830579.402186:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830579.402187:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830579.402191:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4946 00000100:00080000:3.0:1787830579.402193:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830579.402197:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830579.402200:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830579.402209:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671477096576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830579.402218:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830579.402221:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830579.402223:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830579.402226:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4946 00000100:00080000:3.0:1787830579.402230:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830579.402233:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830579.402237:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830579.402245:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671477096704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787830579.402338:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830579.402353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477096192:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830579.402410:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830579.402465:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830579/real 1787830579] req@ffff9a78a6776580 x1874671477096192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830579.402482:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477096192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830579.402491:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830579.402495:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830579.402499:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477096192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830579.402505:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830579.402524:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830579.402535:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830579/real 1787830579] req@ffff9a78a6776940 x1874671477096320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830579.402546:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477096320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830579.402610:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830579.402615:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830579.402618:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830579.402624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830579.402645:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830579.402656:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830579/real 1787830579] req@ffff9a78a67761c0 x1874671477096448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830579.402668:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477096448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830579.402676:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830579.402679:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830579.402682:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830579.402688:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830579.402706:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830579.402717:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830579/real 1787830579] req@ffff9a78a67770c0 x1874671477096576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830579.402728:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671477096576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830579.402737:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830579.402740:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830579.402744:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830579.402749:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830579.402765:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830579.402775:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830579/real 1787830579] req@ffff9a78a6775680 x1874671477096704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830579.402784:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671477096704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830579.402797:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830579.402800:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830579.402802:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830580.426018:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830581.450181:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830582.475306:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830583.498795:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830584.522384:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830584.522393:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830584.522397:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830584.522406:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4952 00000100:00080000:0.0:1787830584.522415:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830584.522421:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830584.522500:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477096832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830584.522531:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830584.522534:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830584.522536:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830584.522540:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4952 00000100:00080000:0.0:1787830584.522543:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830584.522547:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830584.522555:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830584.522566:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477096960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830584.522573:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830584.522576:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830584.522577:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830584.522581:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4952 00000100:00080000:0.0:1787830584.522583:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830584.522585:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830584.522589:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830584.522596:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477097088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830584.522604:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830584.522607:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830584.522608:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830584.522611:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4952 00000100:00080000:0.0:1787830584.522614:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830584.522617:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830584.522621:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830584.522628:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477097216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830584.522636:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830584.522639:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830584.522640:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830584.522644:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4952 00000100:00080000:0.0:1787830584.522647:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830584.522650:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830584.522653:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830584.522662:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477097344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830584.522725:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830584.522783:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477096832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830584.522950:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830584.522976:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830584.522995:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830584.523012:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830585.546590:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830586.570025:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830587.594439:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787830587.595674:0:10925:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:0.0:1787830587.595726:0:10925:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787830587.595742:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830587.595779:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830587.595789:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830587.595797:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:0.0:1787830587.595804:0:10925:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787830587.596483:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830584/real 1787830587] req@ffff9a7890a3c3c0 x1874671477096832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830587.596532:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477096832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830587.596543:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830587.596547:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830587.596552:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477096832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830587.596559:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830584/real 1787830587] req@ffff9a7890a3da40 x1874671477096960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830587.596569:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477096960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830587.596576:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830587.596579:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830587.596582:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477096960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830587.596589:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830584/real 1787830587] req@ffff9a7890a3f480 x1874671477097088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830587.596600:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477097088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830587.596607:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830587.596610:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830587.596613:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830587.596619:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830584/real 1787830587] req@ffff9a7890a3de00 x1874671477097216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830587.596641:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477097216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830587.596650:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830587.596653:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830587.596656:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830587.596662:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830584/real 1787830587] req@ffff9a7890a3fc00 x1874671477097344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830587.596672:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477097344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830587.596680:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830587.596683:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830587.596686:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830588.620638:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830589.641968:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830589.641979:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830589.641984:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830589.641994:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4957 00000100:00080000:2.0:1787830589.642004:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830589.642010:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830589.642054:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477097472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830589.642078:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830589.642084:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830589.642085:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830589.642089:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4957 00000100:00080000:2.0:1787830589.642092:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830589.642096:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830589.642105:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830589.642115:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477097600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830589.642124:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830589.642127:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830589.642128:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830589.642132:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4957 00000100:00080000:2.0:1787830589.642135:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830589.642138:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830589.642143:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830589.642152:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477097728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830589.642172:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830589.642175:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830589.642176:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830589.642180:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4957 00000100:00080000:2.0:1787830589.642182:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830589.642186:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830589.642191:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830589.642200:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477097856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830589.642208:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830589.642211:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830589.642212:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830589.642216:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4957 00000100:00080000:2.0:1787830589.642231:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830589.642236:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830589.642241:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830589.642253:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477097984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787830589.642566:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830589.642927:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477097472:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830589.642985:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830589.644436:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830589/real 1787830589] req@ffff9a78841d8000 x1874671477097472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830589.644463:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477097472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830589.644474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830589.644478:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830589.645591:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477097472:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830589.645606:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097600:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830589.645671:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830589.645696:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830589/real 1787830589] req@ffff9a78841da580 x1874671477097600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830589.645710:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477097600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830589.645720:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830589.645724:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830589.645727:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830589.645733:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097728:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830589.645754:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830589.645766:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830589/real 1787830589] req@ffff9a78841db480 x1874671477097728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830589.645778:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477097728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830589.645792:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830589.645795:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830589.645798:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830589.645805:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097856:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830589.645825:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830589.645855:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830589/real 1787830589] req@ffff9a78841d83c0 x1874671477097856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830589.645866:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477097856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830589.645875:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830589.645878:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830589.645881:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830589.645887:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097984:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830589.645905:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830589.645916:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830589/real 1787830589] req@ffff9a78841db0c0 x1874671477097984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830589.645927:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477097984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830589.645941:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830589.645945:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830589.645947:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477097984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830590.666169:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830591.690049:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830592.714046:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830593.738911:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830594.762152:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830594.762172:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830594.762176:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830594.762196:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4962 00000100:00080000:3.0:1787830594.762205:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830594.762212:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:0.0:1787830594.762235:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830594.762256:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477098112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830594.762292:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830594.762296:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830594.762298:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830594.762301:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4962 00000100:00080000:3.0:1787830594.762306:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830594.762310:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830594.762320:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830594.762333:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477098240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830594.762342:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477098112:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830594.762347:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830594.762351:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830594.762353:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830594.762358:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4962 00000100:00080000:3.0:1787830594.762362:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830594.762366:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830594.762372:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830594.762386:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671477098368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830594.762395:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830594.762399:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830594.762400:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1787830594.762406:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787830594.762409:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4962 00000100:00080000:3.0:1787830594.762414:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830594.762419:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830594.762425:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830594.762438:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671477098496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830594.762447:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830594.762450:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830594.762452:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830594.762456:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4962 00000100:00080000:3.0:1787830594.762460:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830594.762464:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830594.762469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830594/real 1787830594] req@ffff9a78a6774780 x1874671477098112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:3.0:1787830594.762470:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830594.762484:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671477098624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830594.762493:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477098112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830594.762516:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830594.762521:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830594.762525:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477098112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830594.762550:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830594.762572:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830594.762584:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830594/real 1787830594] req@ffff9a78a6776d00 x1874671477098240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830594.762596:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477098240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830594.762604:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830594.762608:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830594.762611:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830594.762616:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830594.762631:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830594.762638:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830594/real 1787830594] req@ffff9a78a6775e00 x1874671477098368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830594.762644:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671477098368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830594.762649:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830594.762651:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830594.762653:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830594.762656:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830594.762665:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830594.762669:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830594/real 1787830594] req@ffff9a78a6777c00 x1874671477098496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830594.762676:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671477098496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830594.762681:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830594.762683:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830594.762685:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830594.762688:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830594.762696:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830594.762701:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830594/real 1787830594] req@ffff9a78a6777840 x1874671477098624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830594.762707:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671477098624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830594.762712:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830594.762714:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830594.762717:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830595.786021:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830596.810512:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830597.835973:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830598.858202:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830599.882223:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830599.882231:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830599.882234:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830599.882244:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4967 00000100:00080000:0.0:1787830599.882252:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830599.882257:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830599.882292:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477098752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830599.882325:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830599.882329:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830599.882331:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787830599.882332:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830599.882337:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4967 00000100:00080000:0.0:1787830599.882341:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830599.882346:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830599.882355:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830599.882369:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477098880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830599.882378:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830599.882381:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830599.882382:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830599.882386:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4967 00000100:00080000:0.0:1787830599.882388:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830599.882391:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830599.882394:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830599.882403:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477099008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830599.882410:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830599.882412:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830599.882413:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830599.882417:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4967 00000100:00080000:0.0:1787830599.882419:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830599.882421:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830599.882423:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477098752:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787830599.882424:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830599.882433:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477099136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830599.882439:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830599.882441:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830599.882442:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830599.882444:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4967 00000100:00080000:0.0:1787830599.882446:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830599.882448:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830599.882450:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830599.882457:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477099264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787830599.882492:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830599.882545:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830599/real 1787830599] req@ffff9a7890a3c000 x1874671477098752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830599.882563:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477098752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830599.882573:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830599.882577:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830599.882580:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477098752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830599.882587:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098880:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830599.882608:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830599.882620:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830599/real 1787830599] req@ffff9a7890a3fc00 x1874671477098880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830599.882631:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477098880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830599.882638:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830599.882641:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830599.882644:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477098880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830599.882665:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830599.882685:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830599.882696:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830599/real 1787830599] req@ffff9a7890a3de00 x1874671477099008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830599.882707:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477099008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830599.882714:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830599.882717:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830599.882720:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830599.882725:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830599.882743:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830599.882754:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830599/real 1787830599] req@ffff9a7890a3f480 x1874671477099136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830599.882770:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477099136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830599.882779:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830599.882782:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830599.882784:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830599.882789:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830599.882807:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830599.882817:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830599/real 1787830599] req@ffff9a7890a3da40 x1874671477099264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830599.882934:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477099264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830599.882944:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830599.882947:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830599.882950:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830600.906110:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830601.930064:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830602.954110:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830603.978346:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830605.002065:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830605.002074:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830605.002079:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830605.002091:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4972 00000100:00080000:0.0:1787830605.002101:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830605.002109:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830605.002153:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477099392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830605.002157:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830605.002194:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830605.002197:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830605.002199:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830605.002202:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4972 00000100:00080000:0.0:1787830605.002205:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830605.002209:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830605.002223:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830605.002236:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477099520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830605.002244:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830605.002246:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830605.002248:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830605.002251:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4972 00000100:00080000:0.0:1787830605.002258:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830605.002261:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830605.002266:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830605.002276:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477099648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830605.002288:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830605.002290:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830605.002292:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830605.002295:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4972 00000100:00080000:0.0:1787830605.002297:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830605.002300:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830605.002304:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830605.002314:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477099776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830605.002321:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830605.002324:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830605.002325:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830605.002328:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4972 00000100:00080000:0.0:1787830605.002331:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830605.002334:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830605.002338:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830605.002346:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477099904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830605.003394:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477099392:192.168.203.140@tcp:250:kworker.0 00000800:00000100:2.0:1787830605.003451:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830605.003503:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830605/real 1787830605] req@ffff9a7890a3c3c0 x1874671477099392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830605.003519:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477099392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830605.003538:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830605.003542:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830605.003546:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477099392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830605.003553:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830605.003571:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830605.003581:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830605/real 1787830605] req@ffff9a7890a3da40 x1874671477099520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830605.003591:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477099520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830605.003599:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830605.003602:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830605.003605:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830605.003610:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830605.003622:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830605.003631:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830605/real 1787830605] req@ffff9a7890a3f480 x1874671477099648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830605.003641:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477099648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830605.003649:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830605.003651:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830605.003655:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830605.003660:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830605.003675:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830605.003683:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830605/real 1787830605] req@ffff9a7890a3de00 x1874671477099776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830605.003694:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477099776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830605.003703:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830605.003706:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830605.003709:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830605.003714:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830605.003725:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830605.003733:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830605/real 1787830605] req@ffff9a7890a3fc00 x1874671477099904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830605.003743:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477099904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830605.003751:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830605.003754:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830605.003756:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477099904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830606.026160:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830607.050925:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830608.074335:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830609.098026:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830610.122035:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830610.124334:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830610.124342:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830610.124347:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830610.124356:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4977 00000100:00080000:2.0:1787830610.124365:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830610.124371:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830610.124403:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477100032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830610.124433:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830610.124437:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830610.124439:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830610.124442:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4977 00000100:00080000:2.0:1787830610.124445:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830610.124449:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830610.124457:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830610.124467:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477100160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830610.124476:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830610.124479:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830610.124480:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830610.124484:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4977 00000100:00080000:2.0:1787830610.124486:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830610.124489:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830610.124492:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830610.124500:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477100288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830610.124514:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830610.124517:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830610.124519:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830610.124522:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4977 00000100:00080000:2.0:1787830610.124524:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830610.124527:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830610.124531:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830610.124539:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477100416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830610.124546:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830610.124549:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830610.124551:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830610.124554:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4977 00000100:00080000:2.0:1787830610.124556:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830610.124559:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830610.124562:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830610.124570:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477100544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830610.124654:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477100032:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830610.124708:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830610.124752:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830610/real 1787830610] req@ffff9a78841da1c0 x1874671477100032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830610.124768:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477100032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830610.124778:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830610.124782:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830610.124785:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477100032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830610.124791:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830610.124812:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830610.124824:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830610/real 1787830610] req@ffff9a78841d9a40 x1874671477100160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830610.124858:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477100160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830610.124866:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830610.124869:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830610.124872:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830610.124878:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830610.124895:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830610.124905:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830610/real 1787830610] req@ffff9a78841d8780 x1874671477100288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830610.124916:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477100288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830610.124924:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830610.124927:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830610.124930:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830610.124935:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830610.124948:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830610.124957:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830610/real 1787830610] req@ffff9a78841d9e00 x1874671477100416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830610.124967:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477100416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830610.124976:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830610.124978:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830610.124981:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830610.124985:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830610.125000:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830610.125009:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830610/real 1787830610] req@ffff9a78841dbc00 x1874671477100544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830610.125019:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477100544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830610.125027:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830610.125030:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830610.125033:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830611.146467:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830612.170115:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830613.194530:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830614.218508:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830615.242089:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830615.242192:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830615.242198:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830615.242201:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830615.242210:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4982 00000100:00080000:2.0:1787830615.242219:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830615.242226:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830615.242260:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477100672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830615.242331:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477100672:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830615.242393:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830615.242424:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830615.242428:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830615.242430:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830615.242433:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4982 00000100:00080000:2.0:1787830615.242437:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830615.242441:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830615.242449:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830615.242461:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477100800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830615.242826:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830615/real 1787830615] req@ffff9a78841dbc00 x1874671477100672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830615.242886:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477100672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830615.242894:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830615.242898:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830615.242902:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477100672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830615.242926:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830615.242961:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830615.242973:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830615/real 1787830615] req@ffff9a78841d9e00 x1874671477100800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830615.242984:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477100800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830615.242992:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830615.242995:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830615.242998:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100800:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830615.243252:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830615.243256:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830615.243258:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830615.243262:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4982 00000100:00080000:2.0:1787830615.243267:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830615.243272:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830615.243278:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830615.243291:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477100928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830615.243301:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830615.243304:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830615.243305:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830615.243309:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4982 00000100:00080000:2.0:1787830615.243311:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787830615.243474:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830615.243638:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830615.243650:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830615/real 1787830615] req@ffff9a78841d8780 x1874671477100928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830615.243667:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477100928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830615.243675:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830615.243678:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830615.243681:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477100928:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830615.243718:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830615.243723:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830615.243734:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477101056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830615.243750:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830615.243753:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830615.243755:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830615.243759:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4982 00000100:00080000:2.0:1787830615.243762:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830615.243765:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830615.243768:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830615.243776:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477101184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830615.243826:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830615.243908:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830615.243918:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830615/real 1787830615] req@ffff9a78841d9a40 x1874671477101056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830615.243929:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477101056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830615.243938:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830615.243941:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830615.243944:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830615.243949:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830615.243962:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830615.243970:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830615/real 1787830615] req@ffff9a78841da1c0 x1874671477101184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830615.243979:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477101184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830615.243993:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830615.243996:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830615.243999:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830616.266095:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830617.290568:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830618.314084:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830619.339445:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830620.362198:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830620.362250:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830620.362267:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830620.362271:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830620.362280:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4987 00000100:00080000:2.0:1787830620.362289:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830620.362296:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830620.362331:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477101312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830620.362356:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830620.362359:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830620.362361:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830620.362364:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4987 00000100:00080000:2.0:1787830620.362367:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830620.362370:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830620.362380:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830620.362391:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477101440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830620.362400:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830620.362403:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830620.362405:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830620.362408:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4987 00000100:00080000:2.0:1787830620.362411:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830620.362414:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830620.362418:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830620.362428:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477101568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830620.362436:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830620.362439:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830620.362441:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830620.362444:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4987 00000100:00080000:2.0:1787830620.362447:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830620.362451:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830620.362455:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830620.362463:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477101696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830620.362471:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830620.362474:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830620.362475:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830620.362479:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4987 00000100:00080000:2.0:1787830620.362481:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830620.362484:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830620.362488:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830620.362502:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477101824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830620.362572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477101312:192.168.203.140@tcp:250:kworker.0 00000800:00000100:1.0:1787830620.362626:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830620.362684:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830620/real 1787830620] req@ffff9a78841db0c0 x1874671477101312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830620.362715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477101312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830620.362724:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830620.362728:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830620.362732:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477101312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:1.0:1787830620.362739:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830620.362760:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830620.362774:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830620/real 1787830620] req@ffff9a78841d83c0 x1874671477101440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830620.362782:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477101440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830620.362787:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830620.362789:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830620.362791:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830620.362795:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830620.362806:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830620.362812:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830620/real 1787830620] req@ffff9a78841db480 x1874671477101568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830620.362818:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477101568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830620.362822:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830620.362824:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830620.362826:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830620.362861:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830620.362874:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830620.362880:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830620/real 1787830620] req@ffff9a78841da580 x1874671477101696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830620.362887:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477101696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830620.362893:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830620.362895:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830620.362897:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830620.362901:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830620.362910:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830620.362925:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830620/real 1787830620] req@ffff9a78841d8000 x1874671477101824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830620.362935:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477101824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830620.362943:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830620.362946:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830620.362949:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477101824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830621.386028:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830622.410775:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830623.437402:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830624.458404:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830625.482151:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830625.482167:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830625.482171:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830625.482182:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4992 00000100:00080000:0.0:1787830625.482193:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830625.482200:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830625.482237:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477101952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830625.482306:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477101952:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830625.482308:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:2.0:1787830625.482379:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830625.482412:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830625.482416:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830625.482418:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830625.482422:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4992 00000100:00080000:0.0:1787830625.482427:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830625.482431:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830625.482441:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830625.482465:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477102080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830625.482636:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830625/real 1787830625] req@ffff9a7890a3c000 x1874671477101952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830625.482653:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477101952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830625.482663:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830625.482667:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830625.482671:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477101952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830625.482990:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830625.483018:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830625.483030:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830625/real 1787830625] req@ffff9a7890a3fc00 x1874671477102080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830625.483042:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477102080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830625.483050:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830625.483053:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830625.483055:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102080:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830625.483270:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830625.483275:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830625.483277:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830625.483282:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4992 00000100:00080000:0.0:1787830625.483287:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830625.483291:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830625.483297:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830625.483310:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477102208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830625.483320:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830625.483324:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830625.483325:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830625.483328:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4992 00000100:00080000:0.0:1787830625.483331:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830625.483334:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830625.483339:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830625.483348:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477102336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830625.483357:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830625.483359:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830625.483361:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830625.483365:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4992 00000100:00080000:0.0:1787830625.483367:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830625.483371:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830625.483376:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830625.483386:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477102464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830625.483479:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830625.483509:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830625.483522:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830625/real 1787830625] req@ffff9a7890a3de00 x1874671477102208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830625.483533:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477102208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830625.483553:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830625.483557:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830625.483560:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830625.483566:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830625.483586:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830625.483598:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830625/real 1787830625] req@ffff9a7890a3f480 x1874671477102336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830625.483607:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477102336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830625.483616:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830625.483619:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830625.483622:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830625.483627:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830625.483644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830625.483655:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830625/real 1787830625] req@ffff9a7890a3da40 x1874671477102464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830625.483665:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477102464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830625.483674:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830625.483677:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830625.483680:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830626.506681:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830627.530060:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830628.554103:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830629.578470:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830630.601990:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830630.602000:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830630.602005:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830630.602014:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 4998 00000100:00080000:2.0:1787830630.602024:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830630.602031:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830630.602071:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477102592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830630.602103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787830630.602106:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830630.602108:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830630.602112:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830630.602116:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4998 00000100:00080000:2.0:1787830630.602121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830630.602125:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830630.602135:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830630.602148:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477102720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830630.602158:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787830630.602161:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477102592:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830630.602162:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830630.602166:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830630.602173:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4998 00000100:00080000:2.0:1787830630.602177:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830630.602180:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830630.602185:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830630.602198:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477102848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830630.602207:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830630.602211:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830630.602213:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830630.602217:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4998 00000100:00080000:2.0:1787830630.602221:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787830630.602223:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830630.602227:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830630.602234:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830630.602245:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477102976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830630.602254:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830630.602257:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830630.602259:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830630.602264:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 4998 00000100:00080000:2.0:1787830630.602268:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830630.602273:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830630.602273:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830630/real 1787830630] req@ffff9a78841d92c0 x1874671477102592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:2.0:1787830630.602280:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830630.602295:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477103104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830630.602305:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477102592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830630.602317:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830630.602322:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830630.602326:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477102592:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830630.602345:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830630.602366:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830630.602377:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830630/real 1787830630] req@ffff9a78841db840 x1874671477102720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830630.602388:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477102720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830630.602394:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830630.602397:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830630.602400:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830630.602405:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830630.602421:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830630.602431:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830630/real 1787830630] req@ffff9a78841d9680 x1874671477102848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830630.602441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477102848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830630.602453:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830630.602456:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830630.602459:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830630.602463:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830630.602482:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830630.602492:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830630/real 1787830630] req@ffff9a78841d8b40 x1874671477102976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830630.602502:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477102976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830630.602511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830630.602515:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830630.602518:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477102976:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830630.602523:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830630.602541:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830630.602552:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830630/real 1787830630] req@ffff9a78841da940 x1874671477103104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830630.602562:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477103104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830630.602570:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830630.602573:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830630.602576:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830631.626818:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830632.650412:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830633.674346:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830634.698271:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830635.722314:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830635.722646:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830635.722652:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830635.722656:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830635.722666:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5003 00000100:00080000:1.0:1787830635.722675:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830635.722683:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830635.722732:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477103232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830635.722792:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477103232:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830635.722876:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830635.722937:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830635.722941:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830635.722943:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830635.722957:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5003 00000100:00080000:1.0:1787830635.722962:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830635.722966:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830635.722976:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830635.722990:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477103360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830635.723318:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830635/real 1787830635] req@ffff9a7890b1da40 x1874671477103232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830635.723335:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477103232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830635.723345:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830635.723349:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830635.723352:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477103232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830635.723374:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830635.723409:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830635.723421:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830635/real 1787830635] req@ffff9a7890b1d680 x1874671477103360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830635.723432:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477103360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830635.723439:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830635.723443:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830635.723446:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103360:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830635.723576:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830635.723580:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830635.723582:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830635.723586:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5003 00000100:00080000:1.0:1787830635.723590:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830635.723594:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830635.723599:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830635.723611:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477103488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830635.723621:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830635.723624:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830635.723625:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830635.723629:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5003 00000100:00080000:1.0:1787830635.723632:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830635.723635:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830635.723639:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830635.723649:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477103616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830635.723656:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830635.723659:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830635.723660:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830635.723663:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5003 00000100:00080000:1.0:1787830635.723666:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830635.723670:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830635.723674:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830635.723681:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477103744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830635.723728:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830635.723751:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830635.723763:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830635/real 1787830635] req@ffff9a7890b1de00 x1874671477103488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830635.723775:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477103488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830635.723782:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830635.723785:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830635.723789:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830635.723794:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830635.723813:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830635.723861:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830635/real 1787830635] req@ffff9a7890b1e580 x1874671477103616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830635.723874:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477103616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830635.723882:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830635.723885:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830635.723887:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830635.723892:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830635.723922:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830635.723932:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830635/real 1787830635] req@ffff9a7890b1e940 x1874671477103744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830635.723942:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477103744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830635.723950:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830635.723953:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830635.723956:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477103744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830636.747703:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830637.770057:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830638.794479:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830639.818107:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830640.842229:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830640.842238:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830640.842242:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830640.842251:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5008 00000100:00080000:0.0:1787830640.842270:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830640.842277:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830640.842310:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477103872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830640.842336:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830640.842339:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830640.842341:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830640.842344:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5008 00000100:00080000:0.0:1787830640.842347:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830640.842350:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830640.842359:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830640.842370:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477104000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830640.842377:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830640.842380:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830640.842382:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830640.842385:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5008 00000100:00080000:0.0:1787830640.842388:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830640.842392:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830640.842397:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787830640.842399:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830640.842407:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477104128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830640.842417:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830640.842420:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830640.842422:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830640.842425:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5008 00000100:00080000:0.0:1787830640.842428:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830640.842430:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830640.842434:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830640.842441:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477104256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830640.842448:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830640.842450:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830640.842452:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830640.842455:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5008 00000100:00080000:0.0:1787830640.842457:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830640.842460:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830640.842463:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830640.842470:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477104384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830640.842536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477103872:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830640.842587:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830640.842627:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830640/real 1787830640] req@ffff9a7890a3c3c0 x1874671477103872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830640.842642:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477103872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830640.842652:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830640.842656:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830640.842660:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477103872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830640.842666:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830640.842682:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830640.842692:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830640/real 1787830640] req@ffff9a7890a3da40 x1874671477104000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830640.842701:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477104000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830640.842708:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830640.842711:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830640.842713:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104000:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830640.842718:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830640.842732:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830640.842741:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830640/real 1787830640] req@ffff9a7890a3f480 x1874671477104128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830640.842752:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477104128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830640.842759:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830640.842762:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830640.842765:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830640.842770:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830640.842790:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830640.842799:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830640/real 1787830640] req@ffff9a7890a3de00 x1874671477104256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830640.842809:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477104256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830640.842817:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830640.842820:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830640.842822:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104256:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830640.842827:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830640.842860:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830640.842882:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830640/real 1787830640] req@ffff9a7890a3fc00 x1874671477104384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830640.842894:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477104384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830640.842904:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830640.842906:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830640.842909:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830641.866057:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830642.890155:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830643.914306:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830644.938180:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830645.963643:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830645.963670:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830645.963675:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830645.963685:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5013 00000100:00080000:2.0:1787830645.963695:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830645.963702:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830645.963737:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477104512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830645.963759:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830645.963762:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830645.963764:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830645.963767:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5013 00000100:00080000:2.0:1787830645.963771:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830645.963775:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830645.963784:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830645.963796:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477104640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830645.963804:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830645.963807:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830645.963808:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830645.963812:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5013 00000100:00080000:2.0:1787830645.963815:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830645.963818:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830645.963822:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830645.963858:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477104768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830645.963867:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830645.963870:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830645.963871:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830645.963875:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5013 00000100:00080000:2.0:1787830645.963878:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830645.963881:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830645.963890:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830645.963900:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477104896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830645.963909:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830645.963911:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830645.963913:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830645.963916:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5013 00000100:00080000:2.0:1787830645.963919:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830645.963922:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830645.963926:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830645.963936:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477105024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830645.972358:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477104512:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830645.972427:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830645.972472:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830645/real 1787830645] req@ffff9a78841da940 x1874671477104512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830645.972489:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477104512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830645.972498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830645.972503:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830645.972507:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477104512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830645.972514:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830645.972534:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830645.972546:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830645/real 1787830645] req@ffff9a78841d8b40 x1874671477104640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830645.972556:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477104640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830645.972568:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830645.972571:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830645.972574:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830645.972580:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830645.972599:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830645.972609:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830645/real 1787830645] req@ffff9a78841d9680 x1874671477104768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830645.972620:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477104768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830645.972627:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830645.972630:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830645.972632:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830645.972638:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830645.972657:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830645.972668:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830645/real 1787830645] req@ffff9a78841db840 x1874671477104896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830645.972679:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477104896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830645.972688:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830645.972691:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830645.972694:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477104896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830645.972700:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830645.972717:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830645.972727:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830645/real 1787830645] req@ffff9a78841d92c0 x1874671477105024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830645.972736:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477105024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830670 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830645.972744:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830645.972746:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830645.972749:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830645.972913:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830646.986076:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830648.010009:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830649.034241:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830650.061883:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830651.082003:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830651.082065:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830651.082072:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830651.082077:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830651.082086:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5018 00000100:00080000:1.0:1787830651.082096:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830651.082103:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830651.082139:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477105152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830651.082201:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477105152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830651.082342:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830651.082347:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830651.082349:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830651.082353:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5018 00000100:00080000:1.0:1787830651.082357:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830651.082362:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830651.082370:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830651.082383:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477105280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830651.082494:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105280:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830651.082598:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830651.082602:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830651.082603:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830651.082607:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5018 00000100:00080000:1.0:1787830651.082611:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830651.082615:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830651.082620:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830651.082631:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477105408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830651.082668:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105408:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830651.082705:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830651.082708:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830651.082710:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830651.082714:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5018 00000100:00080000:1.0:1787830651.082718:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830651.082722:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830651.082731:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830651.082742:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477105536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830651.082775:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105536:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787830651.082918:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830651.082922:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830651.082924:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830651.082928:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5018 00000100:00080000:1.0:1787830651.082932:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830651.082936:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830651.082941:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830651.082953:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477105664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830651.082990:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830652.106178:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830653.140751:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787830654.154105:0:10923:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787830654.154146:0:10923:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787830654.154161:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830654.154195:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830654.154205:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830654.154214:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830654.154223:0:10923:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00010000:00010000:1.0:1787830654.154232:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830654.154292:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830651/real 1787830654] req@ffff9a7890b1fc00 x1874671477105664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830654.155562:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477105664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830654.155584:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830654.155589:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830654.155593:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105664:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830654.155601:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830651/real 1787830654] req@ffff9a7890b1f480 x1874671477105536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830654.155611:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477105536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830654.155619:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830654.155621:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830654.155624:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830654.155629:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830651/real 1787830654] req@ffff9a7890b1c000 x1874671477105408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830654.155639:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477105408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830654.155645:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830654.155648:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830654.155650:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830654.155656:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830651/real 1787830654] req@ffff9a7890b1f840 x1874671477105280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830654.155665:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477105280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830654.155672:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830654.155675:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830654.155678:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830654.155683:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830651/real 1787830654] req@ffff9a7890b1f0c0 x1874671477105152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830654.155693:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477105152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830654.155712:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830654.155715:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830654.155718:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477105152:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830655.180214:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830656.202359:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830656.202455:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830656.202466:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830656.202469:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830656.202479:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5023 00000100:00080000:0.0:1787830656.202489:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830656.202494:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830656.202536:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477105792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830656.202602:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477105792:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830656.202671:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830656.202699:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830656.202869:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830656.202871:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830656.202876:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5023 00000100:00080000:0.0:1787830656.202880:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830656.202885:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830656.202905:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830656.202921:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477105920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830656.203053:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830656/real 1787830656] req@ffff9a7890a3c000 x1874671477105792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830656.203071:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477105792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830656.203081:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830656.203085:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830656.203089:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477105792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830656.203110:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105920:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830656.203137:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830656.203150:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830656/real 1787830656] req@ffff9a7890a3fc00 x1874671477105920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830656.203162:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477105920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830656.203170:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830656.203173:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830656.203176:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477105920:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830656.205057:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830656.205067:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830656.205071:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830656.205079:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5023 00000100:00080000:0.0:1787830656.205088:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830656.205095:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830656.205106:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830656.205134:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477106048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830656.205147:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830656.205151:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830656.205152:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830656.205156:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5023 00000100:00080000:0.0:1787830656.205159:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830656.205163:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830656.205169:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830656.205178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477106176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830656.205188:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830656.205191:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830656.205193:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830656.205197:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5023 00000100:00080000:0.0:1787830656.205199:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830656.205202:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830656.205207:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830656.205217:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477106304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830656.205312:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106048:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830656.205366:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830656.205388:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830656/real 1787830656] req@ffff9a7890a3de00 x1874671477106048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830656.205402:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477106048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830656.205412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830656.205415:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830656.205419:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830656.205424:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106176:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830656.205443:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830656.205454:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830656/real 1787830656] req@ffff9a7890a3f480 x1874671477106176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830656.205470:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477106176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830656.205478:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830656.205480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830656.205483:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830656.205489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106304:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830656.205506:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830656.205517:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830656/real 1787830656] req@ffff9a7890a3da40 x1874671477106304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830656.205528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477106304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830656.205537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830656.205540:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830656.205542:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830657.226504:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830658.253079:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830659.274054:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830660.298193:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830661.322244:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830661.327348:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830661.327359:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830661.327363:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830661.327372:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5028 00000100:00080000:2.0:1787830661.327382:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830661.327389:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830661.327425:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477106432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830661.327463:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830661.327467:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830661.327469:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830661.327473:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5028 00000100:00080000:2.0:1787830661.327476:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830661.327480:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830661.327489:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830661.327502:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477106560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830661.327511:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830661.327514:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830661.327516:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830661.327519:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5028 00000100:00080000:2.0:1787830661.327522:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830661.327525:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830661.327530:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830661.327541:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477106688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830661.327549:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830661.327551:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830661.327553:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830661.327557:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5028 00000100:00080000:2.0:1787830661.327560:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830661.327564:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830661.327569:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830661.327577:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477106816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830661.327585:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830661.327587:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830661.327589:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830661.327593:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5028 00000100:00080000:2.0:1787830661.327595:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830661.327598:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830661.327603:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830661.327612:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477106944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830661.328802:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477106432:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830661.328885:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830661.328942:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830661/real 1787830661] req@ffff9a78841d8000 x1874671477106432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830661.328959:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477106432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830661.328975:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830661.328979:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830661.328982:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477106432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830661.328989:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830661.329012:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830661.329099:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830661/real 1787830661] req@ffff9a78841da580 x1874671477106560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830661.329112:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477106560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830661.329120:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830661.329123:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830661.329126:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830661.329132:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830661.329202:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830661.329215:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830661/real 1787830661] req@ffff9a78841db480 x1874671477106688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830661.329225:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477106688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830661.329233:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830661.329236:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830661.329240:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830661.329245:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830661.329265:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830661.329276:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830661/real 1787830661] req@ffff9a78841d83c0 x1874671477106816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830661.329286:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477106816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830661.329294:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830661.329297:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830661.329300:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830661.329305:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830661.329323:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830661.329334:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830661/real 1787830661] req@ffff9a78841db0c0 x1874671477106944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830661.329344:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477106944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830661.329354:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830661.329356:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830661.329359:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477106944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830662.345991:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830663.373536:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830664.395855:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830665.418076:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830666.441936:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830666.441947:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830666.441951:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830666.441961:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5033 00000100:00080000:2.0:1787830666.441971:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830666.441979:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830666.442013:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477107072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830666.442046:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830666.442049:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830666.442051:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830666.442055:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5033 00000100:00080000:2.0:1787830666.442058:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830666.442062:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830666.442071:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830666.442083:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477107200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830666.442091:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830666.442095:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830666.442096:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830666.442100:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5033 00000100:00080000:2.0:1787830666.442104:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830666.442107:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830666.442111:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830666.442120:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477107328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830666.442128:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830666.442131:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830666.442133:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830666.442136:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5033 00000100:00080000:2.0:1787830666.442139:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830666.442142:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830666.442147:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830666.442155:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477107456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830666.442163:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830666.442165:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830666.442167:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830666.442170:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5033 00000100:00080000:2.0:1787830666.442173:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830666.442176:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830666.442180:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830666.442190:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477107584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830666.442319:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477107072:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830666.442351:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:3.0:1787830666.442484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830666.442544:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830666/real 1787830666] req@ffff9a78841da1c0 x1874671477107072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830666.442561:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477107072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830666.442571:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830666.442575:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830666.442579:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477107072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830666.442586:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830666.442630:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830666.442642:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830666/real 1787830666] req@ffff9a78841d9a40 x1874671477107200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830666.442655:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477107200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830666.442662:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830666.442665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830666.442668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830666.442674:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830666.442702:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830666.442714:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830666/real 1787830666] req@ffff9a78841d8780 x1874671477107328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830666.442726:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477107328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830666.442739:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830666.442742:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830666.442745:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830666.442752:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830666.442784:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830666.442796:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830666/real 1787830666] req@ffff9a78841d9e00 x1874671477107456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830666.442807:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477107456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830666.442816:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830666.442819:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830666.442822:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830666.442827:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830666.442866:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830666.442877:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830666/real 1787830666] req@ffff9a78841dbc00 x1874671477107584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830666.442887:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477107584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830666.442895:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830666.442898:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830666.442901:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830667.466125:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830668.490144:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830669.514171:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830670.539048:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830671.562096:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830671.562108:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830671.562113:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830671.562125:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5039 00000100:00080000:0.0:1787830671.562137:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830671.562145:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830671.562192:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477107712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830671.562274:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477107712:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830671.562358:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830671.562444:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830671.562448:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830671.562450:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830671.562454:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5039 00000100:00080000:0.0:1787830671.562459:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830671.562464:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830671.562475:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830671.562490:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477107840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830671.562591:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830671/real 1787830671] req@ffff9a7890a3c3c0 x1874671477107712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830671.562608:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477107712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830671.562619:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830671.562623:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830671.562628:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477107712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830671.562665:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830671.562689:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830671.562702:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830671/real 1787830671] req@ffff9a7890a3da40 x1874671477107840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830671.562713:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477107840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830671.562720:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830671.562723:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830671.562726:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107840:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830671.562778:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830671.562782:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830671.562784:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830671.562789:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5039 00000100:00080000:0.0:1787830671.562793:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830671.562892:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830671.562898:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830671.562912:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477107968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830671.562923:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830671.562926:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830671.562928:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830671.562932:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5039 00000100:00080000:0.0:1787830671.562937:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830671.562941:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830671.562945:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830671.562955:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477108096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830671.562969:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830671.562972:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830671.562973:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830671.562977:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5039 00000100:00080000:0.0:1787830671.562979:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830671.562982:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830671.562986:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830671.562996:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477108224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830671.563056:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830671.563080:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830671.563093:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830671/real 1787830671] req@ffff9a7890a3f480 x1874671477107968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830671.563103:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477107968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830671.563111:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830671.563113:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830671.563117:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477107968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830671.563122:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830671.563139:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830671.563148:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830671/real 1787830671] req@ffff9a7890a3de00 x1874671477108096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830671.563159:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477108096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830671.563168:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830671.563171:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830671.563174:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830671.563178:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830671.563193:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830671.563201:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830671/real 1787830671] req@ffff9a7890a3fc00 x1874671477108224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830671.563210:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477108224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830671.563219:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830671.563222:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830671.563224:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830671.564668:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830672.586137:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830673.610020:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830674.634015:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830675.658368:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830676.681999:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830676.682009:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830676.682013:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830676.682023:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5044 00000100:00080000:1.0:1787830676.682033:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830676.682040:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830676.682078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477108352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830676.682103:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830676.682107:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830676.682109:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830676.682118:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5044 00000100:00080000:1.0:1787830676.682121:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830676.682126:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830676.682136:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830676.682162:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477108480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830676.682176:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830676.682180:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830676.682182:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830676.682186:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5044 00000100:00080000:1.0:1787830676.682190:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830676.682195:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830676.682199:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830676.682286:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477108608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830676.682298:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830676.682301:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830676.682303:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787830676.683245:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830676.688136:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5044 00000100:00080000:1.0:1787830676.688152:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830676.688161:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830676.688174:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830676.688215:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477108736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830676.688232:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830676.688236:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830676.688238:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830676.688243:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5044 00000100:00080000:1.0:1787830676.688247:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830676.688251:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830676.688256:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830676.688269:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477108864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830676.688494:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477108352:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830676.688556:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830676.688634:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830676/real 1787830676] req@ffff9a7890b1f0c0 x1874671477108352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830676.688651:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477108352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830676.688660:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830676.688665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830676.688668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477108352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830676.688677:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830676.688699:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830676.688710:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830676/real 1787830676] req@ffff9a7890b1f840 x1874671477108480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830676.688721:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477108480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830676.688729:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830676.688732:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830676.688735:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830676.688758:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830676.688777:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830676.688789:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830676/real 1787830676] req@ffff9a7890b1c000 x1874671477108608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830676.688800:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477108608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830676.688808:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830676.688810:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830676.688814:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830676.688819:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830676.688853:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830676.688864:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830676/real 1787830676] req@ffff9a7890b1f480 x1874671477108736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830676.688874:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477108736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830676.688882:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830676.688885:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830676.688887:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830676.688892:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830676.688906:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830676.688915:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830676/real 1787830676] req@ffff9a7890b1fc00 x1874671477108864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830676.688924:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477108864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830676.688932:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830676.688935:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830676.688937:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477108864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830677.706046:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830678.730340:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830679.754645:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830680.778614:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830681.801997:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830681.802008:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830681.802012:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830681.802022:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5049 00000100:00080000:1.0:1787830681.802032:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830681.802039:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830681.802077:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477108992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830681.802102:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830681.802105:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830681.802107:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830681.802110:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5049 00000100:00080000:1.0:1787830681.802113:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830681.802117:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830681.802126:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830681.802138:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477109120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830681.802147:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830681.802150:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830681.802152:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830681.802155:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5049 00000100:00080000:1.0:1787830681.802158:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830681.802161:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830681.802165:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830681.802174:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477109248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830681.802187:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830681.802190:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830681.802192:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830681.802195:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5049 00000100:00080000:1.0:1787830681.802198:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830681.802201:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830681.802206:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830681.802337:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477109376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830681.802347:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830681.802350:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830681.802352:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830681.802355:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5049 00000100:00080000:1.0:1787830681.802359:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830681.802362:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830681.802366:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830681.802375:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477109504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830681.802953:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477108992:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830681.803197:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830681.803254:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830681/real 1787830681] req@ffff9a7890b1e940 x1874671477108992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830681.803272:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477108992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830681.803282:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830681.803287:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830681.803290:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477108992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830681.803298:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830681.803325:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830681.803337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830681/real 1787830681] req@ffff9a7890b1e580 x1874671477109120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830681.803349:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477109120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830681.803362:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830681.803366:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830681.803368:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830681.803374:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830681.803392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830681.803403:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830681/real 1787830681] req@ffff9a7890b1de00 x1874671477109248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830681.803412:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477109248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830681.803419:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830681.803422:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830681.803425:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830681.803430:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830681.803448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830681.803458:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830681/real 1787830681] req@ffff9a7890b1d680 x1874671477109376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830681.803468:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477109376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830681.803477:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830681.803480:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830681.803484:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830681.803489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830681.803507:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830681.803518:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830681/real 1787830681] req@ffff9a7890b1da40 x1874671477109504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830681.803529:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477109504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830681.803539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830681.803542:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830681.803544:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830681.803763:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830682.826046:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830683.850437:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830684.874074:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830685.898134:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830686.922717:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830686.922780:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830686.922786:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830686.922789:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830686.922798:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5054 00000100:00080000:0.0:1787830686.922807:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830686.922813:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830686.922888:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477109632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830686.922919:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830686.922923:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830686.922925:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830686.922929:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5054 00000100:00080000:0.0:1787830686.922932:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830686.922936:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830686.922953:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830686.922965:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477109760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830686.922975:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830686.922978:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830686.922980:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830686.922983:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5054 00000100:00080000:0.0:1787830686.922986:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830686.922990:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830686.922994:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830686.923003:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477109888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830686.923012:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830686.923015:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830686.923017:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830686.923020:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5054 00000100:00080000:0.0:1787830686.923023:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830686.923026:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830686.923031:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830686.923040:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477110016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830686.923049:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830686.923051:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830686.923054:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830686.923057:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5054 00000100:00080000:0.0:1787830686.923060:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830686.923063:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830686.923068:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830686.923077:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477110144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830686.923133:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477109632:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830686.923187:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830686.923240:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830686/real 1787830686] req@ffff9a7890a3c000 x1874671477109632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830686.923257:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477109632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830686.923277:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830686.923282:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830686.923286:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477109632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830686.923292:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830686.923312:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830686.923323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830686/real 1787830686] req@ffff9a7890a3fc00 x1874671477109760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830686.923333:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477109760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830686.923341:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830686.923344:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830686.923347:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830686.923353:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830686.923370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830686.923380:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830686/real 1787830686] req@ffff9a7890a3de00 x1874671477109888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830686.923391:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477109888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830686.923398:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830686.923402:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830686.923405:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477109888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830686.923410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830686.923428:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830686.923439:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830686/real 1787830686] req@ffff9a7890a3f480 x1874671477110016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830686.923449:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477110016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830686.923457:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830686.923460:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830686.923463:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110016:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830686.923468:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830686.923485:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830686.923496:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830686/real 1787830686] req@ffff9a7890a3da40 x1874671477110144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830686.923503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477110144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830686.923509:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830686.923511:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830686.923513:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830687.946171:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830688.970372:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830689.995538:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830691.018344:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830692.042091:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830692.042103:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830692.042108:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830692.042120:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5059 00000100:00080000:1.0:1787830692.042132:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830692.042140:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830692.042189:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477110272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830692.042272:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477110272:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830692.042350:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830692.042384:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830692.042389:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830692.042391:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830692.042395:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5059 00000100:00080000:1.0:1787830692.042399:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830692.042404:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830692.042416:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830692.042431:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477110400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830692.042545:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830692/real 1787830692] req@ffff9a7890b1da40 x1874671477110272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830692.042564:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477110272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830692.042574:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830692.042578:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830692.042582:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477110272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830692.042620:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830692.042644:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830692.042658:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830692/real 1787830692] req@ffff9a7890b1d680 x1874671477110400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830692.042669:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477110400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830692.042700:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830692.042704:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830692.042708:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110400:192.168.203.140@tcp:38:kworker.0 00010000:00010000:2.0:1787830692.042751:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830692.042755:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830692.042760:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830692.042762:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830692.042767:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5059 00000100:00080000:1.0:1787830692.042772:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830692.042776:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830692.042783:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830692.042797:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477110528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830692.042806:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830692.042810:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830692.042812:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830692.042815:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5059 00000100:00080000:1.0:1787830692.042818:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830692.042822:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830692.042827:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830692.042868:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477110656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830692.042876:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830692.042880:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830692.042881:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830692.042885:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5059 00000100:00080000:1.0:1787830692.042888:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830692.042891:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830692.042896:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830692.042905:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477110784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830692.043022:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830692.043047:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830692.043059:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830692/real 1787830692] req@ffff9a7890b1de00 x1874671477110528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830692.043070:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477110528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830692.043078:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830692.043081:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830692.043084:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830692.043089:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830692.043108:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830692.043119:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830692/real 1787830692] req@ffff9a7890b1e580 x1874671477110656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830692.043130:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477110656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830692.043138:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830692.043141:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830692.043145:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830692.043150:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830692.043167:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830692.043177:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830692/real 1787830692] req@ffff9a7890b1e940 x1874671477110784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830692.043187:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477110784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830692.043197:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830692.043200:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830692.043203:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477110784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830693.066210:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830694.091606:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830695.114059:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830696.138031:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830697.161998:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830697.162008:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830697.162013:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830697.162022:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5064 00000100:00080000:0.0:1787830697.162033:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830697.162039:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830697.162074:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477110912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830697.162099:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830697.162102:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830697.162104:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830697.162107:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5064 00000100:00080000:0.0:1787830697.162110:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830697.162114:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830697.162123:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830697.162135:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477111040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830697.162143:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830697.162146:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830697.162148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830697.162151:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5064 00000100:00080000:0.0:1787830697.162154:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830697.162157:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830697.162161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830697.162170:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477111168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830697.162178:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830697.162181:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830697.162182:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830697.162186:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5064 00000100:00080000:0.0:1787830697.162189:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830697.162192:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830697.162196:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830697.162204:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477111296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830697.162211:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830697.162214:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830697.162216:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830697.162219:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5064 00000100:00080000:0.0:1787830697.162222:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830697.162225:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830697.162228:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830697.162236:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477111424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830697.163572:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477110912:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830697.163633:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:3.0:1787830697.163636:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830697.163697:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830697/real 1787830697] req@ffff9a7890a3c3c0 x1874671477110912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830697.163715:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477110912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830697.163725:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830697.163729:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830697.163733:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477110912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830697.163741:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830697.163773:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830697.163786:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830697/real 1787830697] req@ffff9a7890a3da40 x1874671477111040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830697.163798:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477111040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830697.163805:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830697.163808:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830697.163812:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830697.163818:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830697.163853:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830697.163864:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830697/real 1787830697] req@ffff9a7890a3f480 x1874671477111168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830697.163875:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477111168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830697.163883:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830697.163886:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830697.163889:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830697.163895:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830697.163913:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830697.163924:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830697/real 1787830697] req@ffff9a7890a3de00 x1874671477111296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830697.163935:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477111296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830697.163944:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830697.163947:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830697.163950:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830697.163956:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830697.163973:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830697.163983:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830697/real 1787830697] req@ffff9a7890a3fc00 x1874671477111424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830697.163993:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477111424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830697.164004:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830697.164007:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830697.164011:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830698.186556:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830699.210083:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830700.234036:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830701.258021:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830702.281993:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830702.283202:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830702.283210:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830702.283213:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830702.283222:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5069 00000100:00080000:1.0:1787830702.283232:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830702.283240:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830702.283276:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477111552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830702.283303:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830702.283307:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830702.283309:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830702.283312:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5069 00000100:00080000:1.0:1787830702.283316:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830702.283320:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830702.283328:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830702.283339:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477111680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830702.283339:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477111552:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830702.283349:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830702.283351:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830702.283353:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830702.283364:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5069 00000100:00080000:1.0:1787830702.283367:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830702.283371:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830702.283375:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830702.283383:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477111808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830702.283391:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:3.0:1787830702.283392:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830702.283394:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830702.283395:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830702.283398:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5069 00000100:00080000:1.0:1787830702.283401:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830702.283403:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830702.283408:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830702.283416:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477111936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830702.283423:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830702.283426:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830702.283428:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830702.283431:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5069 00000100:00080000:1.0:1787830702.283444:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830702.283448:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830702.283452:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830702.283456:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830702/real 1787830702] req@ffff9a7890b1fc00 x1874671477111552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830702.283462:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477112064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830702.283472:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477111552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830702.283490:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830702.283494:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830702.283498:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477111552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830702.283521:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830702.283543:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830702.283556:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830702/real 1787830702] req@ffff9a7890b1f480 x1874671477111680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830702.283564:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477111680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830702.283572:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830702.283575:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830702.283578:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830702.283584:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830702.283602:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830702.283613:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830702/real 1787830702] req@ffff9a7890b1c000 x1874671477111808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830702.283623:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477111808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830702.283630:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830702.283633:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830702.283636:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111808:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830702.283641:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830702.283659:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830702.283670:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830702/real 1787830702] req@ffff9a7890b1f840 x1874671477111936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830702.283682:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477111936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830702.283690:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830702.283693:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830702.283697:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477111936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830702.283702:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830702.283719:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830702.283729:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830702/real 1787830702] req@ffff9a7890b1f0c0 x1874671477112064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830702.283740:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477112064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830702.283749:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830702.283752:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830702.283755:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830703.307975:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830704.331024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830705.354022:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830706.378340:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830707.401962:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830707.401973:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830707.401978:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830707.401990:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5074 00000100:00080000:1.0:1787830707.402003:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830707.402011:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830707.402049:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477112192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830707.402073:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830707.402077:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830707.402079:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830707.402084:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5074 00000100:00080000:1.0:1787830707.402088:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830707.402092:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830707.402099:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477112192:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787830707.402104:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830707.402121:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477112320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830707.402132:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830707.402135:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830707.402137:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830707.402141:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5074 00000100:00080000:1.0:1787830707.402160:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830707.402165:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787830707.402171:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:1.0:1787830707.402173:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830707.402189:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477112448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787830707.402194:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830707.402201:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830707.402205:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830707.402207:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830707.402213:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5074 00000100:00080000:1.0:1787830707.402217:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830707.402220:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830707/real 1787830707] req@ffff9a7890b1f0c0 x1874671477112192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830707.402224:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830707.402233:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830707.402243:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477112192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830707.402248:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477112576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830707.402265:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830707.402269:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830707.402273:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:1.0:1787830707.402275:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830707.402280:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787830707.402280:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477112192:192.168.203.140@tcp:250:kworker.0 00000100:00080000:1.0:1787830707.402289:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5074 00000100:00080000:1.0:1787830707.402294:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830707.402299:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830707.402304:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830707.402309:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830707.402318:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477112704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830707.402334:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830707.402348:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830707/real 1787830707] req@ffff9a7890b1f840 x1874671477112320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830707.402361:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477112320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830707.402368:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830707.402371:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830707.402374:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112320:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830707.402379:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830707.402419:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830707.402430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830707/real 1787830707] req@ffff9a7890b1c000 x1874671477112448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830707.402441:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477112448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830707.402454:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830707.402458:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830707.402461:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830707.402466:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830707.402484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830707.402495:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830707/real 1787830707] req@ffff9a7890b1f480 x1874671477112576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830707.402505:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477112576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830707.402514:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830707.402518:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830707.402521:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112576:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830707.402536:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830707.402553:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830707.402564:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830707/real 1787830707] req@ffff9a7890b1fc00 x1874671477112704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830707.402575:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477112704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830707.402583:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830707.402586:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830707.402590:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830708.425992:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830709.450039:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830710.474024:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830711.498305:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830712.521987:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830712.521995:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830712.521999:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830712.522008:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5080 00000100:00080000:3.0:1787830712.522017:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830712.522022:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830712.522055:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67743c0 x1874671477112832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830712.522080:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830712.522083:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830712.522085:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830712.522088:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5080 00000100:00080000:3.0:1787830712.522091:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830712.522096:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830712.522105:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830712.522117:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477112832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830712.522120:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774f00 x1874671477112960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830712.522132:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830712.522135:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830712.522137:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830712.522140:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5080 00000100:00080000:3.0:1787830712.522144:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830712.522148:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830712.522153:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830712.522163:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777840 x1874671477113088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830712.522172:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830712.522175:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830712.522178:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1787830712.522179:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:3.0:1787830712.522185:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5080 00000100:00080000:3.0:1787830712.522190:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830712.522194:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830712.522199:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830712.522210:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777c00 x1874671477113216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830712.522218:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787830712.522223:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830712/real 1787830712] req@ffff9a78a67743c0 x1874671477112832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830712.522223:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830712.522227:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830712.522236:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5080 00000100:00080000:3.0:1787830712.522241:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787830712.522244:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67743c0 x1874671477112832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830712.522248:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00080000:1.0:1787830712.522256:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000008:00100000:3.0:1787830712.522257:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00080000:1.0:1787830712.522261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830712.522265:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477112832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830712.522273:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775e00 x1874671477113344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830712.522285:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830712.522305:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830712.522317:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830712/real 1787830712] req@ffff9a78a6774f00 x1874671477112960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830712.522328:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774f00 x1874671477112960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830712.522335:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830712.522338:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830712.522342:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477112960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830712.522347:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:1.0:1787830712.522380:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830712.522391:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830712/real 1787830712] req@ffff9a78a6777840 x1874671477113088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830712.522404:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777840 x1874671477113088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787830712.522411:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830712.522412:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830712.522418:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830712.522422:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830712.522429:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830712.522448:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830712.522469:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830712/real 1787830712] req@ffff9a78a6777c00 x1874671477113216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830712.522490:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777c00 x1874671477113216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830712.522498:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830712.522501:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830712.522504:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830712.522530:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:1.0:1787830712.522547:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:1.0:1787830712.522555:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830712/real 1787830712] req@ffff9a78a6775e00 x1874671477113344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830712.522567:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775e00 x1874671477113344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830712.522572:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830712.522575:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830712.522577:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830713.546769:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830714.570047:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830715.594643:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830716.618234:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830717.641997:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830717.642006:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830717.642010:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830717.642019:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5085 00000100:00080000:3.0:1787830717.642028:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830717.642035:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830717.642132:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477113472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830717.642200:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477113472:192.168.203.140@tcp:250:kworker.0 00000100:00080000:3.0:1787830717.642386:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830717.642390:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830717.642392:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830717.642397:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5085 00000100:00080000:3.0:1787830717.642401:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830717.642414:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830717.642423:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830717.642437:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477113600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830717.642526:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113600:192.168.203.140@tcp:38:kworker.0 00000100:00080000:3.0:1787830717.643058:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830717.643063:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830717.643065:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830717.643070:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5085 00000100:00080000:3.0:1787830717.643075:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830717.643080:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830717.643086:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830717.643100:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775680 x1874671477113728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830717.643112:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830717.643115:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830717.643117:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830717.643121:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5085 00000100:00080000:3.0:1787830717.643124:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830717.643127:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830717.643132:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830717.643143:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67770c0 x1874671477113856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830717.643153:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830717.643156:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830717.643157:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830717.643161:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5085 00000100:00080000:3.0:1787830717.643164:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830717.643168:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830717.643172:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830717.643181:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477113984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787830717.643283:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830717.643314:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830717.643350:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113984:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830717.643953:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830718.666347:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830719.690263:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830720.714169:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787830720.718255:0:10922:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:3.0:1787830720.718433:0:10922:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787830720.718449:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787830720.718547:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787830720.718558:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787830720.718567:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:3.0:1787830720.718576:0:10922:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:1.0:1787830720.718604:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830717/real 1787830720] req@ffff9a78a6775680 x1874671477113728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830720.718628:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775680 x1874671477113728/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830720.718640:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830720.718644:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830720.718648:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113728:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830720.718657:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830717/real 1787830720] req@ffff9a78a67770c0 x1874671477113856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830720.718669:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67770c0 x1874671477113856/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830720.718678:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830720.718681:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830720.718685:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113856:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830720.718690:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830717/real 1787830720] req@ffff9a78a67761c0 x1874671477113984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830720.718700:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477113984/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830720.718709:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830720.718712:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830720.718714:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113984:192.168.203.140@tcp:8:kworker.0 00000100:00100000:1.0:1787830720.718720:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830717/real 1787830720] req@ffff9a78a6774780 x1874671477113600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830720.718730:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477113600/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830720.718737:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830720.718739:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830720.718742:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477113600:192.168.203.140@tcp:38:kworker.0 00000100:00100000:1.0:1787830720.718759:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830717/real 1787830720] req@ffff9a78a6776d00 x1874671477113472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787830720.718770:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477113472/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787830720.718777:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787830720.718780:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:1.0:1787830720.718782:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477113472:192.168.203.140@tcp:250:kworker.0 00010000:00010000:0.0:1787830721.738751:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830722.761995:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830722.762005:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830722.762010:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830722.762020:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5090 00000100:00080000:2.0:1787830722.762030:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830722.762038:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830722.762072:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477114112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830722.762105:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830722.762109:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830722.762111:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830722.762115:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5090 00000100:00080000:2.0:1787830722.762119:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830722.762124:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830722.762133:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830722.762147:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477114240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830722.762158:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830722.762161:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830722.762163:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787830722.762168:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830722.762168:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5090 00000100:00080000:2.0:1787830722.762175:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830722.762180:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830722.762185:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830722.762196:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477114368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830722.762205:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830722.762209:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830722.762210:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830722.762214:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5090 00000100:00080000:2.0:1787830722.762217:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830722.762220:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830722.762225:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830722.762234:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477114496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830722.762243:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830722.762246:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830722.762247:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830722.762250:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5090 00000100:00080000:2.0:1787830722.762253:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830722.762257:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830722.762261:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830722.762271:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477114624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830722.762427:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477114112:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830722.762486:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830722.762528:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830722/real 1787830722] req@ffff9a78841dbc00 x1874671477114112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830722.762543:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477114112/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830722.762553:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830722.762557:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830722.762561:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477114112:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830722.762568:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114240:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830722.762590:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830722.762602:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830722/real 1787830722] req@ffff9a78841d9e00 x1874671477114240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830722.762614:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477114240/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830722.762621:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830722.762624:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830722.762627:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114240:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830722.762633:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114368:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830722.762653:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830722.762664:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830722/real 1787830722] req@ffff9a78841d8780 x1874671477114368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830722.762674:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477114368/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830722.762681:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830722.762683:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830722.762685:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114368:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830722.762691:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114496:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830722.762710:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830722.762721:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830722/real 1787830722] req@ffff9a78841d9a40 x1874671477114496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830722.762732:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477114496/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830722.762742:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830722.762744:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830722.762747:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114496:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830722.762752:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114624:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830722.762873:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830722.762884:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830722/real 1787830722] req@ffff9a78841da1c0 x1874671477114624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830722.762902:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477114624/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830722.762910:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830722.762913:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830722.762915:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114624:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830723.786277:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830724.810194:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830725.834153:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830726.858339:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830727.882193:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830727.882206:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830727.882212:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830727.882222:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5095 00000100:00080000:2.0:1787830727.882234:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830727.882240:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830727.882307:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477114752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830727.882339:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830727.882347:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830727.882349:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830727.882353:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5095 00000100:00080000:2.0:1787830727.882357:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830727.882362:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830727.882375:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830727.882389:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477114880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830727.882401:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830727.882406:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830727.882411:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787830727.882413:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477114752:192.168.203.140@tcp:250:kworker.0 00000100:00080000:2.0:1787830727.882420:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5095 00000100:00080000:2.0:1787830727.882429:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830727.882434:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830727.882440:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830727.882457:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477115008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830727.882472:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830727.882476:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830727.882478:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830727.882483:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5095 00000100:00080000:2.0:1787830727.882487:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830727.882491:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830727.882496:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000800:00000100:3.0:1787830727.882498:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830727.882516:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477115136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830727.882527:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830727.882531:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830727.882537:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830727.882542:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5095 00010000:00010000:0.0:1787830727.882623:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830727.882628:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830727/real 1787830727] req@ffff9a78841db0c0 x1874671477114752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830727.882650:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477114752/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830727.882660:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830727.882665:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830727.882668:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477114752:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830727.882692:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114880:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830727.883561:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830727.883574:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830727.883582:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830727.883604:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477115264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830727.883619:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830727.883648:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830727/real 1787830727] req@ffff9a78841d83c0 x1874671477114880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830727.883663:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477114880/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830727.883679:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830727.883683:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830727.883686:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477114880:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830727.883693:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115008:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830727.883863:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830727.883883:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830727/real 1787830727] req@ffff9a78841db480 x1874671477115008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830727.883894:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477115008/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830727.883902:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830727.883905:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830727.883914:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115008:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830727.883922:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115136:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830727.883942:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830727.883953:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830727/real 1787830727] req@ffff9a78841da580 x1874671477115136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830727.883964:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477115136/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830727.883973:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830727.883976:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830727.883980:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115136:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830727.884012:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115264:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830727.884031:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830727.884043:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830727/real 1787830727] req@ffff9a78841d8000 x1874671477115264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830727.884054:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477115264/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830727.884063:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830727.884066:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830727.884069:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115264:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830728.909579:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830729.930046:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830730.955153:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830731.979545:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830733.002627:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830733.002636:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830733.002640:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830733.002650:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5100 00000100:00080000:0.0:1787830733.002660:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830733.002667:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830733.002704:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477115392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830733.002736:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830733.002740:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830733.002742:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830733.002746:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5100 00000100:00080000:0.0:1787830733.002750:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830733.002753:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830733.002762:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830733.002776:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477115520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830733.002784:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830733.002787:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830733.002789:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830733.002793:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5100 00000100:00080000:0.0:1787830733.002796:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830733.002805:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830733.002806:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477115392:192.168.203.140@tcp:250:kworker.0 00000008:00100000:0.0:1787830733.002810:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830733.002824:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477115648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830733.002861:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830733.002865:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830733.002867:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830733.002872:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5100 00000800:00000100:2.0:1787830733.002893:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00010000:00010000:1.0:1787830733.004673:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830733.005340:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830733.005350:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830733.005360:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830733.005384:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477115776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830733.005396:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830733.005399:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830733.005401:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830733.005405:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5100 00000100:00080000:0.0:1787830733.005409:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830733.005413:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830733.005417:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830733.005428:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477115904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830733.006610:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830733/real 1787830733] req@ffff9a7890a3c000 x1874671477115392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830733.006663:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477115392/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830733.006680:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830733.006684:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830733.006688:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477115392:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830733.006702:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115520:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830733.006763:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830733.006778:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830733/real 1787830733] req@ffff9a7890a3fc00 x1874671477115520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830733.006789:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477115520/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830733.006803:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830733.006806:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830733.006809:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115520:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830733.006849:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115648:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830733.006873:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830733.006884:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830733/real 1787830733] req@ffff9a7890a3de00 x1874671477115648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830733.006896:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477115648/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830733.006909:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830733.006913:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830733.006916:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115648:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830733.006921:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115776:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830733.006941:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830733.006952:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830733/real 1787830733] req@ffff9a7890a3f480 x1874671477115776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830733.006962:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477115776/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830733.006971:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830733.006974:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830733.006977:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115776:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830733.006982:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115904:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830733.006998:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830733.007008:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830733/real 1787830733] req@ffff9a7890a3da40 x1874671477115904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830733.007019:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477115904/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830733.007027:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830733.007030:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830733.007033:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477115904:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830734.026492:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830735.051380:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830736.074032:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830737.098089:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830738.121944:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830738.121954:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830738.121959:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830738.121969:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5105 00000100:00080000:3.0:1787830738.121979:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830738.121987:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830738.122021:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477116032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830738.122050:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830738.122053:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830738.122055:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830738.122058:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5105 00000100:00080000:3.0:1787830738.122062:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830738.122066:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830738.122074:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830738.122085:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477116160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830738.122093:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830738.122096:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830738.122098:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830738.122101:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5105 00000100:00080000:3.0:1787830738.122104:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830738.122108:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830738.122112:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830738.122120:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477116288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830738.122128:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830738.122131:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830738.122133:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830738.122137:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5105 00000100:00080000:3.0:1787830738.122139:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830738.122142:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830738.122146:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830738.122154:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477116416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830738.122162:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830738.122165:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830738.122166:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830738.122170:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5105 00000100:00080000:3.0:1787830738.122173:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830738.122176:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830738.122180:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830738.122198:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477116544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830738.122292:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787830738.122315:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477116032:192.168.203.140@tcp:250:kworker.0 00000800:00000100:2.0:1787830738.122375:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830738.122418:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830738/real 1787830738] req@ffff9a78a6776940 x1874671477116032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830738.122434:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477116032/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830738.122443:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830738.122447:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830738.122451:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477116032:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830738.122457:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116160:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830738.122476:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830738.122487:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830738/real 1787830738] req@ffff9a78a6776580 x1874671477116160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830738.122498:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477116160/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830738.122505:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830738.122508:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830738.122511:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116160:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830738.122516:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116288:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830738.122531:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830738.122541:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830738/real 1787830738] req@ffff9a78a6774b40 x1874671477116288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830738.122552:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477116288/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830738.122559:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830738.122562:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830738.122565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116288:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830738.122570:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116416:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830738.122586:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830738.122595:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830738/real 1787830738] req@ffff9a78a67752c0 x1874671477116416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830738.122605:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477116416/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830738.122613:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830738.122616:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830738.122619:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116416:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830738.122624:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116544:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830738.122641:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830738.122651:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830738/real 1787830738] req@ffff9a78a6775a40 x1874671477116544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830738.122661:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477116544/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830738.122670:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830738.122673:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830738.122676:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116544:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830739.146230:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830740.170032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830741.194111:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830742.218543:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830743.243080:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830743.243129:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830743.243135:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830743.243138:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830743.243148:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5110 00000100:00080000:2.0:1787830743.243167:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830743.243173:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830743.243205:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477116672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830743.243230:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830743.243234:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830743.243235:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830743.243239:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5110 00000100:00080000:2.0:1787830743.243241:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830743.243245:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830743.243252:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830743.243261:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477116800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830743.243270:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830743.243272:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830743.243274:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830743.243278:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5110 00000100:00080000:2.0:1787830743.243280:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830743.243282:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830743.243286:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830743.243293:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477116928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830743.243300:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830743.243303:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830743.243304:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830743.243307:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5110 00000100:00080000:2.0:1787830743.243309:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830743.243312:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830743.243315:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830743.243322:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477117056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830743.243330:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830743.243333:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830743.243335:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830743.243338:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5110 00000100:00080000:2.0:1787830743.243340:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830743.243343:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830743.243346:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830743.243353:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477117184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830743.243410:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477116672:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830743.243461:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830743.243513:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830743/real 1787830743] req@ffff9a78841d92c0 x1874671477116672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830743.243528:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477116672/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830743.243537:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830743.243541:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830743.243544:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477116672:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830743.243550:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116800:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830743.243567:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830743.243577:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830743/real 1787830743] req@ffff9a78841db840 x1874671477116800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830743.243588:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477116800/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830743.243596:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830743.243599:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830743.243602:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116800:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830743.243606:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116928:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830743.243619:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830743.243627:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830743/real 1787830743] req@ffff9a78841d9680 x1874671477116928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830743.243636:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477116928/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830743.243643:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830743.243646:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830743.243649:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477116928:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830743.243653:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117056:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830743.243672:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830743.243681:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830743/real 1787830743] req@ffff9a78841d8b40 x1874671477117056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830743.243691:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477117056/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830743.243699:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830743.243702:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830743.243705:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117056:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830743.243710:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117184:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830743.243722:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830743.243729:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830743/real 1787830743] req@ffff9a78841da940 x1874671477117184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830743.243739:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477117184/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830743.243756:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830743.243759:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830743.243762:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117184:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830744.265992:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830745.290286:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830746.314576:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830747.338515:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830748.362068:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830748.362078:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830748.362104:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830748.362123:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5115 00000100:00080000:1.0:1787830748.362134:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830748.362140:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830748.362196:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477117312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830748.362232:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830748.362236:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830748.362238:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830748.362242:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5115 00000100:00080000:1.0:1787830748.362246:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830748.362250:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830748.362260:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830748.362282:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477117440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830748.362291:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830748.362295:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830748.362297:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830748.362300:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5115 00000100:00080000:1.0:1787830748.362304:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830748.362307:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830748.362312:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830748.362320:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477117568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830748.362329:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830748.362332:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830748.362334:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830748.362338:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5115 00000100:00080000:1.0:1787830748.362341:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830748.362344:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830748.362349:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830748.362357:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477117696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830748.362365:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830748.362368:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830748.362370:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830748.362374:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5115 00000100:00080000:1.0:1787830748.362377:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830748.362380:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830748.362384:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830748.362393:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477117824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830748.362498:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477117312:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830748.362589:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830748.362655:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830748/real 1787830748] req@ffff9a7890b1e940 x1874671477117312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830748.362675:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477117312/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830748.362687:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830748.362691:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830748.362696:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477117312:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830748.362704:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117440:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830748.362728:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830748.362741:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830748/real 1787830748] req@ffff9a7890b1e580 x1874671477117440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830748.362754:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477117440/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830748.362763:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830748.362767:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830748.362771:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117440:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830748.362777:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117568:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830748.362802:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830748.362818:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830748/real 1787830748] req@ffff9a7890b1de00 x1874671477117568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830748.362871:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477117568/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830748.362880:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830748.362883:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830748.362885:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117568:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830748.362892:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117696:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830748.362913:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830748.362925:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830748/real 1787830748] req@ffff9a7890b1d680 x1874671477117696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830748.362937:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477117696/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830748.362948:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830748.362952:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830748.362956:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117696:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830748.362962:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117824:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830748.362982:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830748.362993:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830748/real 1787830748] req@ffff9a7890b1da40 x1874671477117824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830748.363005:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477117824/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830748.363015:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830748.363018:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830748.363022:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477117824:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830748.363149:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830749.386276:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830750.411446:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830751.435194:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830752.458154:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830753.481948:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830753.481958:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830753.481963:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830753.481973:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5120 00000100:00080000:1.0:1787830753.481983:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830753.481989:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830753.482025:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477117952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830753.482053:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830753.482056:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830753.482058:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830753.482062:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5120 00000100:00080000:1.0:1787830753.482065:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830753.482069:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830753.482085:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830753.482100:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477118080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830753.482109:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830753.482112:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830753.482114:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830753.482119:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5120 00000100:00080000:1.0:1787830753.482122:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830753.482126:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830753.482130:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787830753.482138:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787830753.482148:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477118208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830753.482158:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830753.482161:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830753.482162:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830753.482166:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5120 00000100:00080000:1.0:1787830753.482169:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830753.482173:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830753.482177:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830753.482186:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477118336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830753.482194:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830753.482196:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830753.482198:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830753.482202:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5120 00000100:00080000:1.0:1787830753.482205:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830753.482208:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830753.482212:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830753.482221:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477118464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830753.482289:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477117952:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830753.482341:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830753.482379:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830753/real 1787830753] req@ffff9a7890b1da40 x1874671477117952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830753.482393:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477117952/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830753.482402:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830753.482406:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830753.482409:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477117952:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830753.482415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118080:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830753.482432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830753.482442:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830753/real 1787830753] req@ffff9a7890b1d680 x1874671477118080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830753.482451:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477118080/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830753.482458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830753.482461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830753.482464:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118080:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830753.482469:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118208:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830753.482484:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830753.482493:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830753/real 1787830753] req@ffff9a7890b1de00 x1874671477118208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830753.482503:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477118208/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830753.482511:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830753.482514:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830753.482517:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118208:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830753.482522:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118336:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830753.482540:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830753.482551:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830753/real 1787830753] req@ffff9a7890b1e580 x1874671477118336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830753.482561:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477118336/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830753.482582:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830753.482585:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830753.482588:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118336:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830753.482594:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118464:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830753.482613:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830753.482623:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830753/real 1787830753] req@ffff9a7890b1e940 x1874671477118464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830753.482634:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477118464/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830753.482642:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830753.482645:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830753.482648:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118464:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830754.506060:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830755.530061:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830756.554225:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830757.578010:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830758.602355:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830758.602365:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830758.602369:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830758.602378:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5126 00000100:00080000:1.0:1787830758.602386:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830758.602392:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830758.602433:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477118592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830758.602463:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830758.602467:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830758.602468:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830758.602472:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5126 00000100:00080000:1.0:1787830758.602476:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830758.602479:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830758.602580:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477118592:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830758.602636:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830758.602680:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830758/real 1787830758] req@ffff9a7890b1fc00 x1874671477118592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830758.602696:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477118592/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830758.602706:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830758.602710:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830758.602713:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477118592:192.168.203.140@tcp:250:kworker.0 00000008:00100000:1.0:1787830758.602816:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830758.602851:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477118720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830758.602871:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830758.602874:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830758.602876:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830758.602880:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5126 00000100:00080000:1.0:1787830758.602884:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830758.602888:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830758.602893:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830758.602904:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477118848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830758.602913:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830758.602916:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830758.602918:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830758.602921:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5126 00000100:00080000:1.0:1787830758.602924:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830758.602927:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830758.602931:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830758.602940:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477118976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830758.602947:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830758.602949:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830758.602951:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830758.602953:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5126 00000100:00080000:1.0:1787830758.602956:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830758.602958:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830758.602961:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830758.602969:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477119104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830758.603148:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118720:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830758.603172:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830758.603184:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830758/real 1787830758] req@ffff9a7890b1f480 x1874671477118720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830758.603195:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477118720/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830758.603203:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830758.603206:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830758.603209:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118720:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830758.603214:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118848:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830758.603233:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830758.603241:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830758/real 1787830758] req@ffff9a7890b1c000 x1874671477118848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830758.603251:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477118848/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830758.603258:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830758.603261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830758.603264:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118848:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830758.603269:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118976:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830758.603286:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830758.603297:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830758/real 1787830758] req@ffff9a7890b1f840 x1874671477118976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830758.603310:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477118976/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830758.603319:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830758.603322:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830758.603326:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477118976:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830758.603328:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787830758.603332:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119104:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830758.603352:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830758.603362:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830758/real 1787830758] req@ffff9a7890b1f0c0 x1874671477119104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830758.603372:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477119104/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830758.603381:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830758.603384:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830758.603387:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119104:192.168.203.140@tcp:8:kworker.0 00010000:00010000:2.0:1787830759.626072:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830760.650856:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830761.674008:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830762.699477:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830763.722040:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830763.722051:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830763.722055:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830763.722065:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5131 00000100:00080000:0.0:1787830763.722076:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830763.722083:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830763.722119:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477119232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830763.722142:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830763.722146:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830763.722148:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830763.722151:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5131 00000100:00080000:0.0:1787830763.722154:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830763.722158:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830763.722167:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830763.722178:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477119360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830763.722186:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830763.722189:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830763.722191:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830763.722194:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5131 00000100:00080000:0.0:1787830763.722197:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830763.722199:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830763.722202:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830763.722210:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477119488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830763.722219:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830763.722222:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830763.722223:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830763.722226:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5131 00000100:00080000:0.0:1787830763.722229:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830763.722232:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830763.722236:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830763.722243:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477119616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830763.722251:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830763.722254:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830763.722255:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830763.722258:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5131 00000100:00080000:0.0:1787830763.722260:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830763.722263:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830763.722267:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830763.722274:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477119744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787830763.723968:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477119232:192.168.203.140@tcp:250:kworker.0 00000800:00000100:2.0:1787830763.724054:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830763.724118:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830763/real 1787830763] req@ffff9a7890a3c3c0 x1874671477119232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830763.724138:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477119232/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830763.724149:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830763.724153:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830763.724157:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477119232:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830763.724165:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119360:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830763.724188:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830763.724200:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830763/real 1787830763] req@ffff9a7890a3da40 x1874671477119360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830763.724212:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477119360/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830763.724220:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830763.724223:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830763.724227:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119360:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830763.724234:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119488:192.168.203.140@tcp:38:kworker.0 00000800:00000100:2.0:1787830763.724254:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830763.724264:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830763/real 1787830763] req@ffff9a7890a3f480 x1874671477119488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830763.724275:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477119488/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830763.724294:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830763.724298:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830763.724301:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119488:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830763.724307:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119616:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830763.724326:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830763.724337:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830763/real 1787830763] req@ffff9a7890a3de00 x1874671477119616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830763.724348:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477119616/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830763.724357:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830763.724360:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830763.724363:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119616:192.168.203.140@tcp:8:kworker.0 00000100:00100000:2.0:1787830763.724369:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119744:192.168.203.140@tcp:8:kworker.0 00000800:00000100:2.0:1787830763.724387:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:2.0:1787830763.724410:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830763/real 1787830763] req@ffff9a7890a3fc00 x1874671477119744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787830763.724421:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477119744/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787830763.724431:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787830763.724434:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:2.0:1787830763.724437:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477119744:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830763.724608:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830764.745947:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830765.774153:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830766.796857:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830767.818032:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830768.842004:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830768.842012:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830768.842017:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830768.842035:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5136 00000100:00080000:0.0:1787830768.842045:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830768.842051:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830768.842085:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477119872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830768.842162:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477119872:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830768.842217:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830768.842248:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830768.842252:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830768.842255:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830768.842259:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5136 00010000:00010000:2.0:1787830768.842264:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830768.842265:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830768.842271:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830768.842281:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830768.842295:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477120000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830768.842430:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830768/real 1787830768] req@ffff9a7890a3c000 x1874671477119872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830768.842447:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477119872/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830768.842458:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830768.842461:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830768.842465:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477119872:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830768.842485:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120000:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830768.842506:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830768.842519:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830768/real 1787830768] req@ffff9a7890a3fc00 x1874671477120000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830768.842531:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477120000/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830768.842539:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830768.842542:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830768.842546:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120000:192.168.203.140@tcp:38:kworker.0 00000100:00080000:0.0:1787830768.842684:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830768.842688:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830768.842690:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830768.842695:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5136 00000100:00080000:0.0:1787830768.842699:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830768.842703:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830768.842708:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830768.842722:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477120128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830768.842769:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120128:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830768.842790:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830768.842809:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830768.842813:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830768.842815:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830768.842819:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5136 00000100:00080000:0.0:1787830768.842823:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830768.842827:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830768.842851:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830768.842864:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477120256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830768.842923:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830768/real 1787830768] req@ffff9a7890a3de00 x1874671477120128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830768.842936:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477120128/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830768.842950:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830768.842953:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830768.842956:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120128:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830768.842968:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120256:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830768.842999:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830768.843011:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830768/real 1787830768] req@ffff9a7890a3f480 x1874671477120256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830768.843023:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477120256/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830768.843032:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830768.843035:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830768.843038:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120256:192.168.203.140@tcp:8:kworker.0 00000100:00080000:0.0:1787830768.843328:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830768.843332:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830768.843334:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830768.843339:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5136 00000100:00080000:0.0:1787830768.843343:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830768.843347:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830768.843352:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830768.843364:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477120384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830768.843403:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120384:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830768.843432:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830768.843536:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830768/real 1787830768] req@ffff9a7890a3da40 x1874671477120384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830768.843548:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477120384/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830768.843558:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830768.843561:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830768.843565:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120384:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830769.866244:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830770.890174:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830771.914526:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830772.938064:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830773.961919:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830773.961930:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830773.961934:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830773.961957:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5141 00000100:00080000:0.0:1787830773.961968:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830773.961976:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830773.962012:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477120512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830773.962035:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830773.962039:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830773.962041:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830773.962045:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5141 00000100:00080000:0.0:1787830773.962048:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830773.962052:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830773.962062:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830773.962064:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477120512:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830773.962078:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477120640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830773.962089:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830773.962092:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830773.962094:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830773.962098:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5141 00000100:00080000:0.0:1787830773.962104:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830773.962108:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000800:00000100:3.0:1787830773.962112:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000008:00100000:0.0:1787830773.962114:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830773.962128:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477120768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830773.962137:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830773.962140:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830773.962142:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830773.962146:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5141 00000100:00080000:0.0:1787830773.962149:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830773.962153:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830773.962158:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830773.962161:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830773/real 1787830773] req@ffff9a7890a3c3c0 x1874671477120512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830773.962171:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477120896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830773.962178:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477120512/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830773.962186:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830773.962189:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830773.962190:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830773.962195:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830773.962196:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00080000:0.0:1787830773.962205:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5141 00000100:00100000:3.0:1787830773.962206:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477120512:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830773.962214:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830773.962220:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830773.962225:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830773.962229:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120640:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830773.962245:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830773.962251:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477121024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830773.962259:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830773/real 1787830773] req@ffff9a7890a3da40 x1874671477120640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830773.962273:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477120640/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830773.962282:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830773.962285:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830773.962289:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120640:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830773.962295:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120768:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830773.962312:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830773.962323:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830773/real 1787830773] req@ffff9a7890a3f480 x1874671477120768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830773.962334:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477120768/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830773.962342:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830773.962345:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830773.962348:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120768:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830773.962354:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120896:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830773.962370:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830773.962378:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830773/real 1787830773] req@ffff9a7890a3de00 x1874671477120896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830773.962386:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477120896/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830773.962395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830773.962398:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830773.962401:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477120896:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830773.962415:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121024:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830773.962433:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830773.962443:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830773/real 1787830773] req@ffff9a7890a3fc00 x1874671477121024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830773.962464:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477121024/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830773.962474:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830773.962477:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830773.962481:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121024:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830773.964563:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830774.986029:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830776.010123:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830777.035142:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830778.058131:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830779.082029:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830779.082039:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830779.082044:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830779.082056:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5146 00000100:00080000:0.0:1787830779.082067:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830779.082074:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830779.082116:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477121152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830779.082141:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830779.082144:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830779.082145:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830779.082148:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5146 00000100:00080000:0.0:1787830779.082150:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830779.082152:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830779.082161:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830779.082169:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477121280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830779.082175:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830779.082177:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830779.082178:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830779.082181:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5146 00000100:00080000:0.0:1787830779.082183:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830779.082185:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830779.082188:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830779.082194:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477121408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830779.082200:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830779.082203:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830779.082204:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830779.082208:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5146 00000100:00080000:0.0:1787830779.082212:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787830779.082212:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477121152:192.168.203.140@tcp:250:kworker.0 00000100:00080000:0.0:1787830779.082218:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830779.082224:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830779.082237:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477121536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830779.082246:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830779.082249:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830779.082252:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830779.082257:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5146 00010000:00010000:1.0:1787830779.082260:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830779.082280:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830779.082284:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830779.082290:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000800:00000100:3.0:1787830779.082291:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830779.082306:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477121664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830779.082364:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830779/real 1787830779] req@ffff9a7890a3c000 x1874671477121152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830779.082383:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477121152/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830779.082395:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830779.082399:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830779.082402:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477121152:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830779.082409:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121280:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830779.082430:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830779.082441:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830779/real 1787830779] req@ffff9a7890a3fc00 x1874671477121280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830779.082452:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477121280/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830779.082460:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830779.082463:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830779.082466:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121280:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830779.082492:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121408:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830779.082509:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830779.082518:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830779/real 1787830779] req@ffff9a7890a3de00 x1874671477121408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830779.082776:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477121408/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830779.082785:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830779.082789:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830779.082793:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121408:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830779.082799:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121536:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830779.082822:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830779.082858:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830779/real 1787830779] req@ffff9a7890a3f480 x1874671477121536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830779.082869:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477121536/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830779.082878:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830779.082882:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830779.082885:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121536:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830779.082891:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121664:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830779.082921:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830779.082932:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830779/real 1787830779] req@ffff9a7890a3da40 x1874671477121664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830779.082944:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477121664/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830779.082953:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830779.082956:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830779.082959:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121664:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830780.106023:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787830781.130912:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830782.154094:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830783.178346:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830784.202147:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830784.202158:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830784.202162:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830784.202172:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5151 00000100:00080000:2.0:1787830784.202183:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830784.202191:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830784.202229:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477121792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830784.202268:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830784.202272:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830784.202274:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830784.202279:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5151 00000100:00080000:2.0:1787830784.202283:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830784.202288:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830784.202297:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830784.202299:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477121792:192.168.203.140@tcp:250:kworker.0 00000100:00100000:2.0:1787830784.202313:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477121920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830784.202325:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830784.202328:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830784.202330:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830784.202335:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5151 00000100:00080000:2.0:1787830784.202350:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830784.202356:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830784.202362:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830784.202376:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477122048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830784.202386:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830784.202390:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1787830784.202392:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121920:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830784.202394:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787830784.202398:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830784.202407:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5151 00000100:00080000:2.0:1787830784.202416:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830784.202420:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830784.202425:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122048:192.168.203.140@tcp:38:kworker.0 00000008:00100000:2.0:1787830784.202426:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830784.202444:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db840 x1874671477122176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830784.202463:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830784.202466:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830784.202468:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830784.202474:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5151 00000100:00080000:2.0:1787830784.202478:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787830784.202483:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122176:192.168.203.140@tcp:8:kworker.0 00000100:00080000:2.0:1787830784.202485:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830784.202494:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830784.202507:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d92c0 x1874671477122304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830784.202533:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122304:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830785.226129:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830786.250128:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787830787.277334:0:10924:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.140:988 00000400:00000100:2.0:1787830787.277569:0:10924:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.140@tcp at host 192.168.203.140 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787830787.277584:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830787.277620:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830787.277629:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830787.277637:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000800:00000100:2.0:1787830787.277643:0:10924:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.140@tcp 00000100:00100000:0.0:1787830787.277887:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830784/real 1787830787] req@ffff9a78841d92c0 x1874671477122304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830787.277915:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d92c0 x1874671477122304/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830787.277927:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830787.277932:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830787.277935:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122304:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830787.277942:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830784/real 1787830787] req@ffff9a78841db840 x1874671477122176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830787.277951:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db840 x1874671477122176/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830787.277960:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830787.277962:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830787.277965:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122176:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830787.277971:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830784/real 1787830787] req@ffff9a78841d8b40 x1874671477121920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830787.277980:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477121920/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830787.277986:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830787.277989:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830787.277991:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477121920:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830787.277995:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830784/real 1787830787] req@ffff9a78841d9680 x1874671477122048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830787.278005:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477122048/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830787.278010:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830787.278013:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830787.278015:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122048:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830787.278020:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830784/real 1787830787] req@ffff9a78841da940 x1874671477121792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830787.278029:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477121792/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830787.278035:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830787.278037:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830787.278040:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477121792:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830787.278595:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830788.298082:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830789.322145:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830789.322155:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830789.322158:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830789.322168:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5156 00000100:00080000:2.0:1787830789.322178:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830789.322184:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830789.322216:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477122432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830789.322240:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830789.322243:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830789.322244:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830789.322248:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5156 00000100:00080000:2.0:1787830789.322252:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830789.322255:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830789.322265:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830789.322276:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477122560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830789.322285:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830789.322288:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830789.322290:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830789.322293:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5156 00000100:00080000:2.0:1787830789.322295:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830789.322299:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830789.322303:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830789.322312:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477122688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830789.322320:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830789.322322:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830789.322324:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830789.322327:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5156 00000100:00080000:2.0:1787830789.322330:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830789.322334:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830789.322338:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830789.322347:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477122816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830789.322355:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830789.322358:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830789.322367:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830789.322372:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5156 00000100:00080000:2.0:1787830789.322375:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830789.322379:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830789.322383:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830789.322393:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477122944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830789.322489:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477122432:192.168.203.140@tcp:250:kworker.0 00010000:00010000:1.0:1787830789.324094:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:0.0:1787830789.327606:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830789.327679:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830789/real 1787830789] req@ffff9a78841d8000 x1874671477122432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830789.327701:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477122432/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830789.327711:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830789.327715:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830789.327719:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477122432:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830789.327730:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122560:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830789.327765:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830789.327779:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830789/real 1787830789] req@ffff9a78841da580 x1874671477122560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830789.327791:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477122560/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830789.327799:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830789.327802:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830789.327806:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122560:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830789.327812:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122688:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830789.327854:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830789.327867:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830789/real 1787830789] req@ffff9a78841db480 x1874671477122688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830789.327878:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477122688/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830789.327886:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830789.327889:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830789.327893:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122688:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830789.327899:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122816:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830789.327919:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830789.327930:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830789/real 1787830789] req@ffff9a78841d83c0 x1874671477122816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830789.327941:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477122816/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830789.327951:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830789.327954:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830789.327957:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122816:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830789.327964:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122944:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830789.327982:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830789.327992:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830789/real 1787830789] req@ffff9a78841db0c0 x1874671477122944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830789.328004:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477122944/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830789.328013:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830789.328016:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830789.328019:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477122944:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830790.346813:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830791.372728:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830792.394355:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830793.418487:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830794.442167:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830794.442556:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830794.442563:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830794.442566:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830794.442575:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5161 00000100:00080000:2.0:1787830794.442584:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830794.442590:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830794.442623:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477123072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830794.442651:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830794.442656:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830794.442658:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830794.442661:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5161 00000100:00080000:2.0:1787830794.442664:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830794.442668:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830794.442676:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830794.442686:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477123200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830794.442695:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830794.442697:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830794.442699:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830794.442703:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5161 00000100:00080000:2.0:1787830794.442705:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830794.442708:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830794.442711:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830794.442721:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477123328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830794.442728:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830794.442731:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830794.442733:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830794.442736:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5161 00000100:00080000:2.0:1787830794.442739:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830794.442742:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830794.442746:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830794.442753:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477123456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830794.442761:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830794.442764:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830794.442765:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830794.442767:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5161 00000100:00080000:2.0:1787830794.442770:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830794.442773:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830794.442776:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830794.442783:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477123584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830794.442856:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477123072:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830794.442912:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830794.442956:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830794/real 1787830794] req@ffff9a78841da1c0 x1874671477123072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830794.442973:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477123072/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830794.442982:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830794.442986:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830794.442989:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477123072:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830794.442996:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123200:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830794.443015:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830794.443025:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830794/real 1787830794] req@ffff9a78841d9a40 x1874671477123200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830794.443036:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477123200/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830794.443050:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830794.443053:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830794.443056:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123200:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830794.443060:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123328:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830794.443073:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830794.443081:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830794/real 1787830794] req@ffff9a78841d8780 x1874671477123328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830794.443090:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477123328/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830794.443098:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830794.443100:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830794.443103:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123328:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830794.443108:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123456:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830794.443120:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830794.443128:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830794/real 1787830794] req@ffff9a78841d9e00 x1874671477123456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830794.443138:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477123456/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830794.443146:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830794.443148:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830794.443152:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123456:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830794.443156:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123584:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830794.443167:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830794.443174:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830794/real 1787830794] req@ffff9a78841dbc00 x1874671477123584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830794.443183:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477123584/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830794.443191:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830794.443193:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830794.443196:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123584:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830795.466185:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830796.490326:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830797.514787:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830798.538115:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830799.562434:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830799.562444:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830799.562449:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830799.562458:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5167 00000100:00080000:3.0:1787830799.562467:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830799.562474:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830799.563581:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6777480 x1874671477123712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830799.563615:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830799.563619:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830799.563621:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830799.563625:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5167 00000100:00080000:3.0:1787830799.563630:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830799.563635:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830799.563645:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830799.563661:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774000 x1874671477123840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830799.563670:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830799.563673:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830799.563675:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830799.563678:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5167 00000100:00080000:3.0:1787830799.563681:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830799.563684:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830799.563687:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830799.563698:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6775a40 x1874671477123968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830799.563706:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830799.563709:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830799.563711:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830799.563714:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5167 00000100:00080000:3.0:1787830799.563717:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830799.563720:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830799.563724:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830799.563734:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67752c0 x1874671477124096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830799.563741:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830799.563745:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830799.563746:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830799.563749:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5167 00000100:00080000:3.0:1787830799.563752:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830799.563755:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00010000:00010000:1.0:1787830799.563853:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00100000:3.0:1787830799.565333:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830799.565359:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774b40 x1874671477124224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830799.565864:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477123712:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830799.565925:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830799.565970:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830799/real 1787830799] req@ffff9a78a6777480 x1874671477123712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830799.565984:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6777480 x1874671477123712/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830799.565993:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830799.565997:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830799.566000:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6777480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477123712:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830799.566007:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123840:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830799.566027:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830799.566039:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830799/real 1787830799] req@ffff9a78a6774000 x1874671477123840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830799.566051:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774000 x1874671477123840/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830799.566242:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830799.566247:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830799.566250:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123840:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830799.566256:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123968:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830799.566277:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830799.566289:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830799/real 1787830799] req@ffff9a78a6775a40 x1874671477123968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830799.566300:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6775a40 x1874671477123968/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830799.566307:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830799.566310:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830799.566313:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6775a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477123968:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830799.566320:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124096:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830799.566338:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830799.566350:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830799/real 1787830799] req@ffff9a78a67752c0 x1874671477124096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830799.566361:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67752c0 x1874671477124096/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830799.566369:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830799.566373:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830799.566376:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124096:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830799.566381:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124224:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830799.566399:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830799.566408:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830799/real 1787830799] req@ffff9a78a6774b40 x1874671477124224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830799.566419:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774b40 x1874671477124224/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830799.566427:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830799.566430:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830799.566432:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124224:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830800.586826:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830801.611637:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830802.634557:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830803.658211:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830804.682267:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830804.682277:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830804.682282:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830804.682292:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5172 00000100:00080000:2.0:1787830804.682303:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830804.682311:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830804.682349:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841dbc00 x1874671477124352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830804.682377:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830804.682381:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830804.682383:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830804.682387:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5172 00000100:00080000:2.0:1787830804.682391:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830804.682396:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830804.682405:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830804.682418:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9e00 x1874671477124480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830804.682427:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830804.682430:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830804.682432:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830804.682436:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5172 00000100:00080000:2.0:1787830804.682439:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830804.682442:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830804.682446:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830804.682456:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8780 x1874671477124608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830804.682465:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830804.682468:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830804.682470:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830804.682473:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5172 00000100:00080000:2.0:1787830804.682476:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830804.682480:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830804.682485:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830804.682494:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9a40 x1874671477124736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830804.682508:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830804.682511:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830804.682512:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830804.682527:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5172 00000100:00080000:2.0:1787830804.682531:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830804.682535:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830804.682539:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830804.682550:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da1c0 x1874671477124864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787830804.682687:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787830804.683702:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477124352:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830804.683782:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830804.683857:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830804/real 1787830804] req@ffff9a78841dbc00 x1874671477124352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830804.683875:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841dbc00 x1874671477124352/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830804.683883:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830804.683886:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830804.683889:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841dbc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477124352:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830804.683907:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124480:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830804.683930:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830804.683942:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830804/real 1787830804] req@ffff9a78841d9e00 x1874671477124480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830804.683953:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9e00 x1874671477124480/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830804.683960:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830804.683963:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830804.683965:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124480:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830804.683970:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124608:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830804.683988:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830804.683999:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830804/real 1787830804] req@ffff9a78841d8780 x1874671477124608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830804.684009:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8780 x1874671477124608/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830804.684019:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830804.684023:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830804.684026:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124608:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830804.684031:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124736:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830804.684049:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830804.684060:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830804/real 1787830804] req@ffff9a78841d9a40 x1874671477124736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830804.684070:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9a40 x1874671477124736/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830804.684079:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830804.684082:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830804.684085:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124736:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830804.684091:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124864:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830804.684118:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830804.684130:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830804/real 1787830804] req@ffff9a78841da1c0 x1874671477124864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830804.684141:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da1c0 x1874671477124864/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830804.684150:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830804.684154:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830804.684157:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477124864:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830805.706058:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830806.730298:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830807.754045:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830808.778914:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830809.802009:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830809.802017:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830809.802021:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830809.802028:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5177 00000100:00080000:1.0:1787830809.802036:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830809.802042:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830809.802080:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f0c0 x1874671477124992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830809.802343:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830809.802347:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830809.802349:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830809.802353:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5177 00000100:00080000:1.0:1787830809.802357:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830809.802362:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830809.802371:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830809.802384:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f840 x1874671477125120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830809.802394:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830809.802397:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830809.802398:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830809.802402:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5177 00000100:00080000:1.0:1787830809.802405:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830809.802408:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830809.802412:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830809.802422:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1c000 x1874671477125248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830809.802431:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830809.802433:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830809.802435:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830809.802439:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5177 00000100:00080000:1.0:1787830809.802442:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830809.802444:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830809.802448:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830809.802469:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1f480 x1874671477125376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830809.802478:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830809.802480:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830809.802482:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830809.802485:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5177 00000100:00080000:1.0:1787830809.802488:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830809.802492:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830809.802496:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830809.802505:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1fc00 x1874671477125504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830809.803011:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477124992:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830809.803069:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830809.803121:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830809/real 1787830809] req@ffff9a7890b1f0c0 x1874671477124992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830809.803136:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f0c0 x1874671477124992/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830809.803146:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830809.803150:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830809.803154:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477124992:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830809.803160:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125120:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830809.803181:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830809.803193:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830809/real 1787830809] req@ffff9a7890b1f840 x1874671477125120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830809.803203:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f840 x1874671477125120/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830809.803211:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830809.803214:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830809.803217:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125120:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830809.803222:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125248:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830809.803239:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830809.803249:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830809/real 1787830809] req@ffff9a7890b1c000 x1874671477125248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830809.803259:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1c000 x1874671477125248/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830809.803266:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830809.803269:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830809.803272:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125248:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830809.803277:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125376:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830809.803293:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830809.803302:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830809/real 1787830809] req@ffff9a7890b1f480 x1874671477125376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830809.803312:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1f480 x1874671477125376/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830809.803321:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830809.803324:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830809.803327:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125376:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830809.803333:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125504:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830809.803349:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830809.803359:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830809/real 1787830809] req@ffff9a7890b1fc00 x1874671477125504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830809.803369:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1fc00 x1874671477125504/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830809.803378:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830809.803380:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830809.803383:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125504:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830809.824660:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830810.826803:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830811.852041:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830812.874030:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830813.908857:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830814.922234:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830814.922290:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830814.922297:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830814.922301:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830814.922310:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5182 00000100:00080000:2.0:1787830814.922320:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830814.922327:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830814.922363:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db0c0 x1874671477125632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830814.922391:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830814.922395:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830814.922397:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830814.922401:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5182 00000100:00080000:2.0:1787830814.922404:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830814.922408:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830814.922419:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830814.922430:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d83c0 x1874671477125760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830814.922440:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830814.922443:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830814.922454:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830814.922459:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5182 00000100:00080000:2.0:1787830814.922463:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830814.922466:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830814.922470:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830814.922571:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477125632:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830814.922628:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830814.922682:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830814/real 1787830814] req@ffff9a78841db0c0 x1874671477125632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830814.922699:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db0c0 x1874671477125632/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830814.922709:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830814.922712:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830814.922715:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477125632:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830814.922721:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125760:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830814.922740:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830814.922751:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830814/real 1787830814] req@ffff9a78841d83c0 x1874671477125760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830814.922762:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d83c0 x1874671477125760/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830814.922770:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830814.922774:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830814.922777:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d83c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125760:192.168.203.140@tcp:38:kworker.0 00000100:00100000:2.0:1787830814.922907:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841db480 x1874671477125888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830814.922959:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125888:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830814.922989:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830814.923010:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830814.923014:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830814.923016:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830814.923020:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5182 00000100:00080000:2.0:1787830814.923023:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830814.923026:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830814.923032:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:2.0:1787830814.923044:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da580 x1874671477126016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830814.925910:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830814/real 1787830814] req@ffff9a78841db480 x1874671477125888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830814.930245:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841db480 x1874671477125888/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830814.930257:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830814.930261:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830814.930265:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477125888:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830814.930297:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126016:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830814.930360:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830814.930383:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830814/real 1787830814] req@ffff9a78841da580 x1874671477126016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830814.930396:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da580 x1874671477126016/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830814.930406:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830814.930409:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830814.930412:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126016:192.168.203.140@tcp:8:kworker.0 00000100:00080000:2.0:1787830814.931328:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830814.931335:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:2.0:1787830814.931339:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830814.931345:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5182 00000100:00080000:2.0:1787830814.931352:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830814.931358:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830814.931368:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:2.0:1787830814.931394:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8000 x1874671477126144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830814.932755:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126144:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830814.932813:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830814.932867:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830814/real 1787830814] req@ffff9a78841d8000 x1874671477126144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830814.932882:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8000 x1874671477126144/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830814.932893:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830814.932897:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830814.932900:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126144:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830815.946131:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830817.133564:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830818.187486:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830819.211153:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830820.042038:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830820.042051:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830820.042055:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830820.042064:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5187 00000100:00080000:1.0:1787830820.042073:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830820.042080:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830820.042401:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477126272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830820.042579:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830820.042583:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830820.042585:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830820.042590:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5187 00000100:00080000:1.0:1787830820.042594:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830820.042599:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830820.042608:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830820.042633:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477126400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830820.042644:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830820.042647:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830820.042649:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830820.042653:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5187 00000100:00080000:1.0:1787830820.042657:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830820.042660:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830820.042665:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830820.042676:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477126528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830820.042686:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830820.042689:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830820.042702:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830820.042706:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5187 00000100:00080000:1.0:1787830820.042709:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830820.042713:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830820.042717:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830820.042728:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477126656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830820.042737:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830820.042740:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830820.042742:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830820.042746:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5187 00000100:00080000:1.0:1787830820.042749:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830820.042753:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830820.042757:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830820.042767:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477126784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830820.043568:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477126272:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830820.043920:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830820.044086:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830820/real 1787830820] req@ffff9a7890b1e940 x1874671477126272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830820.044222:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477126272/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830820.044233:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830820.044248:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830820.044251:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477126272:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830820.044259:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126400:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830820.044284:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830820.044297:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830820/real 1787830820] req@ffff9a7890b1e580 x1874671477126400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830820.045076:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477126400/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830820.045092:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830820.045096:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830820.045099:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126400:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830820.045179:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126528:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830820.045209:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830820.045223:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830820/real 1787830820] req@ffff9a7890b1de00 x1874671477126528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830820.045235:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477126528/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830820.045243:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830820.045246:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830820.045249:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126528:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830820.045256:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126656:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830820.045276:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830820.045287:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830820/real 1787830820] req@ffff9a7890b1d680 x1874671477126656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830820.045297:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477126656/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830820.047186:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830820.047197:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830820.047204:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126656:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830820.047214:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126784:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830820.047258:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830820.047277:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830820/real 1787830820] req@ffff9a7890b1da40 x1874671477126784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830820.047293:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477126784/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830820.047303:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830820.048605:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830820.048612:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477126784:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830820.234305:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830821.258083:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830822.283150:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830823.308723:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830824.330066:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787830825.162106:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830825.162116:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787830825.162120:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830825.162130:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5192 00000100:00080000:0.0:1787830825.162140:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830825.162146:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:0.0:1787830825.162213:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c3c0 x1874671477126912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830825.162244:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830825.162248:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787830825.162250:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830825.162254:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5192 00000100:00080000:0.0:1787830825.162258:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830825.162262:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830825.162272:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830825.162284:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3da40 x1874671477127040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830825.162293:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830825.162296:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787830825.162298:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830825.162302:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5192 00000100:00080000:0.0:1787830825.162304:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830825.162307:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830825.162311:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787830825.162321:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3f480 x1874671477127168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830825.162328:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830825.162331:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787830825.162332:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830825.162335:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5192 00000100:00080000:0.0:1787830825.162337:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830825.162340:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830825.162344:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830825.162352:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3de00 x1874671477127296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787830825.162359:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830825.162362:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830825.162364:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830825.162367:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5192 00000100:00080000:0.0:1787830825.162370:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830825.162373:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830825.162377:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830825.162385:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477127424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830825.162467:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477126912:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830825.162523:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830825.162621:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830825/real 1787830825] req@ffff9a7890a3c3c0 x1874671477126912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830825.162638:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c3c0 x1874671477126912/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830825.162648:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830825.162651:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830825.162656:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477126912:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830825.162663:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127040:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830825.162686:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830825.162698:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830825/real 1787830825] req@ffff9a7890a3da40 x1874671477127040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830825.162710:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3da40 x1874671477127040/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830825.162718:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830825.162720:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830825.162723:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127040:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830825.162729:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127168:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830825.162748:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830825.162759:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830825/real 1787830825] req@ffff9a7890a3f480 x1874671477127168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830825.162770:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3f480 x1874671477127168/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830825.162778:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830825.162781:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830825.162783:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127168:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830825.162800:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127296:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830825.162821:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830825.162861:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830825/real 1787830825] req@ffff9a7890a3de00 x1874671477127296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830825.162872:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3de00 x1874671477127296/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830825.162881:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830825.162884:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830825.162888:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127296:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830825.162894:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127424:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830825.162912:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830825.162923:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830825/real 1787830825] req@ffff9a7890a3fc00 x1874671477127424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830825.162934:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477127424/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830825.162943:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830825.162946:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830825.162950:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127424:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830825.354234:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830826.380788:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830827.402203:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830828.426047:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830829.450689:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787830830.295063:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830830.295085:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787830830.295090:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830830.295099:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5197 00000100:00080000:2.0:1787830830.295111:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830830.295128:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:2.0:1787830830.295163:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841da940 x1874671477127552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787830830.295205:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830830.295209:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787830830.295211:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830830.295215:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5197 00000100:00080000:2.0:1787830830.295218:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830830.295222:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830830.295232:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830830.295253:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d9680 x1874671477127680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830830.295901:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477127552:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830830.295957:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830830.296015:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830830/real 1787830830] req@ffff9a78841da940 x1874671477127552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830830.296031:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841da940 x1874671477127552/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830830.296041:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830830.296045:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830830.296048:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841da940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477127552:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830830.297851:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127680:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830830.297914:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:2.0:1787830830.297965:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830830.297970:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787830830.297972:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830830.297977:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5197 00000100:00080000:2.0:1787830830.297982:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830830.297988:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:2.0:1787830830.297998:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787830830.298021:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78841d8b40 x1874671477127808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830830.298746:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830830/real 1787830830] req@ffff9a78841d9680 x1874671477127680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830830.298764:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d9680 x1874671477127680/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830830.298773:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830830.298776:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830830.298779:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d9680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127680:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830830.298801:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127808:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830830.298854:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830830.298867:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830830/real 1787830830] req@ffff9a78841d8b40 x1874671477127808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830830.298948:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78841d8b40 x1874671477127808/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830830.299600:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830830.299605:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830830.299608:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78841d8b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127808:192.168.203.140@tcp:38:kworker.0 00000100:00080000:2.0:1787830830.299758:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787830830.299763:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1787830830.299765:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787830830.299770:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5197 00000100:00080000:2.0:1787830830.299775:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787830830.299780:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830830.300029:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:0.0:1787830830.300060:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3c000 x1874671477127936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830830.302581:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127936:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830830.302651:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:0.0:1787830830.303487:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787830830.303493:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:0.0:1787830830.303497:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787830830.303504:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5197 00000100:00080000:0.0:1787830830.303510:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787830830.303516:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:0.0:1787830830.303526:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:0.0:1787830830.303563:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890a3fc00 x1874671477128064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830830.303655:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830830/real 1787830830] req@ffff9a7890a3c000 x1874671477127936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830830.303673:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3c000 x1874671477127936/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830830.303686:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830830.303691:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830830.303695:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477127936:192.168.203.140@tcp:8:kworker.0 00000100:00100000:3.0:1787830830.303719:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128064:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830830.303748:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830830.303760:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830830/real 1787830830] req@ffff9a7890a3fc00 x1874671477128064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830830.303771:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890a3fc00 x1874671477128064/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830830.303780:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830830.303783:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830830.303786:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890a3fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128064:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830830.474703:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830831.498486:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830832.523157:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830833.546431:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830834.570391:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830835.402386:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830835.402396:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830835.402431:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830835.402448:0:19259:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5202 00000100:00080000:1.0:1787830835.402458:0:19259:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830835.402465:0:19259:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:1.0:1787830835.402504:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1da40 x1874671477128192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830835.402646:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477128192:192.168.203.140@tcp:250:kworker.0 00000800:00000100:3.0:1787830835.402717:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830835.403395:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830835.403400:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830835.403402:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830835.403406:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5202 00000100:00080000:1.0:1787830835.403411:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830835.403416:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830835.403425:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830835.403440:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1d680 x1874671477128320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830835.403611:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830835/real 1787830835] req@ffff9a7890b1da40 x1874671477128192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830835.403630:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1da40 x1874671477128192/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830835.403640:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830835.403644:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830835.403647:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477128192:192.168.203.140@tcp:250:kworker.0 00000100:00100000:3.0:1787830835.403694:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128320:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830835.403717:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830835.403729:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830835/real 1787830835] req@ffff9a7890b1d680 x1874671477128320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830835.403740:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1d680 x1874671477128320/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830835.403747:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830835.403751:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830835.403754:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128320:192.168.203.140@tcp:38:kworker.0 00000100:00080000:1.0:1787830835.405475:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830835.405482:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830835.405485:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830835.405492:0:19259:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5202 00000100:00080000:1.0:1787830835.405499:0:19259:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830835.405504:0:19259:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830835.405524:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830835.405547:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1de00 x1874671477128448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830835.405593:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128448:192.168.203.140@tcp:38:kworker.0 00000800:00000100:3.0:1787830835.405632:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00080000:1.0:1787830835.405657:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830835.405660:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1787830835.405662:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830835.405666:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5202 00000100:00080000:1.0:1787830835.405669:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830835.405673:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830835.405677:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:1.0:1787830835.405688:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e580 x1874671477128576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830835.405725:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830835/real 1787830835] req@ffff9a7890b1de00 x1874671477128448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830835.405738:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1de00 x1874671477128448/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830835.405747:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830835.405750:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830835.405754:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128448:192.168.203.140@tcp:38:kworker.0 00000100:00100000:3.0:1787830835.405768:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128576:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830835.405786:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830835.405796:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830835/real 1787830835] req@ffff9a7890b1e580 x1874671477128576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830835.405807:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e580 x1874671477128576/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830835.405816:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830835.405818:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830835.405821:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128576:192.168.203.140@tcp:8:kworker.0 00000100:00080000:1.0:1787830835.405962:0:19259:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830835.405966:0:19259:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:1.0:1787830835.405968:0:19259:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830835.405972:0:19259:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5202 00000100:00080000:1.0:1787830835.405976:0:19259:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830835.405979:0:19259:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:1.0:1787830835.405984:0:19259:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:1.0:1787830835.405995:0:19259:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a7890b1e940 x1874671477128704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787830835.406032:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128704:192.168.203.140@tcp:8:kworker.0 00000800:00000100:3.0:1787830835.406052:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:3.0:1787830835.406129:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830835/real 1787830835] req@ffff9a7890b1e940 x1874671477128704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830835.406140:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a7890b1e940 x1874671477128704/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830835.406150:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830835.406152:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:3.0:1787830835.406155:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a7890b1e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128704:192.168.203.140@tcp:8:kworker.0 00010000:00010000:0.0:1787830835.594722:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830836.618048:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830837.642130:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830838.666042:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830839.694338:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787830840.522047:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3510e4c2-7f35-4dff-9360-35db1135010d->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830840.522057:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787830840.522061:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830840.522070:0:11718:0:(import.c:510:import_select_connection()) MGC192.168.203.140@tcp: connect to NID 192.168.203.140@tcp last attempt 5208 00000100:00080000:3.0:1787830840.522080:0:11718:0:(import.c:589:import_select_connection()) MGC192.168.203.140@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830840.522087:0:11718:0:(import.c:624:import_select_connection()) MGC192.168.203.140@tcp: import ffff9a78a6511800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000100:00100000:3.0:1787830840.522124:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776580 x1874671477128832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830840.522156:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830840.522160:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787830840.522162:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830840.522165:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5208 00000100:00080000:3.0:1787830840.522169:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830840.522173:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9a78a6512000: import ffff9a788849e000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830840.522182:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830840.522194:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776940 x1874671477128960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830840.522213:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830840.522217:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787830840.522219:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830840.522223:0:11718:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5208 00000100:00080000:3.0:1787830840.522226:0:11718:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830840.522230:0:11718:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9a78a6512000: import ffff9a788849e800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830840.522234:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683204 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787830840.522245:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6776d00 x1874671477129088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830840.522253:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830840.522257:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787830840.522258:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830840.522262:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5208 00000100:00080000:3.0:1787830840.522265:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830840.522268:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9a78a6512000: import ffff9a788923f800 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830840.522273:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6836224, lost: 0. 00000100:00100000:3.0:1787830840.522282:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a6774780 x1874671477129216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787830840.522290:0:11718:0:(pinger.c:229:ptlrpc_pinger_process_import()) 97196825-2cb9-49ba-a47d-1455a997ce3a->lustre-OST0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787830840.522293:0:11718:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0001_UUID: starting recovery 00000100:00080000:3.0:1787830840.522294:0:11718:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787830840.522298:0:11718:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: connect to NID 192.168.203.140@tcp last attempt 5208 00000100:00080000:3.0:1787830840.522300:0:11718:0:(import.c:589:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787830840.522304:0:11718:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9a78a6512000: import ffff9a788923f000 using connection 192.168.203.140@tcp/192.168.203.140@tcp 00000008:00100000:3.0:1787830840.522308:0:11718:0:(osc_request.c:4019:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683204 ocd_grant: 6889472, lost: 0. 00000100:00100000:3.0:1787830840.522317:0:11718:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9a78a67761c0 x1874671477129344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787830840.525003:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477128832:192.168.203.140@tcp:250:kworker.0 00000800:00000100:0.0:1787830840.525133:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830840.525187:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830840/real 1787830840] req@ffff9a78a6776580 x1874671477128832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830840.525205:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776580 x1874671477128832/t0(0) o250->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830840.525217:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a78a6511800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830840.525221:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830840.525225:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:3510e4c2-7f35-4dff-9360-35db1135010d:10933:1874671477128832:192.168.203.140@tcp:250:kworker.0 00000100:00100000:0.0:1787830840.525232:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128960:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830840.525255:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830840.525267:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830840/real 1787830840] req@ffff9a78a6776940 x1874671477128960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830840.525278:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776940 x1874671477128960/t0(0) o38->lustre-MDT0000-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830840.525285:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830840.525289:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830840.525292:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477128960:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830840.525301:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477129088:192.168.203.140@tcp:38:kworker.0 00000800:00000100:0.0:1787830840.525320:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830840.525330:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830840/real 1787830840] req@ffff9a78a6776d00 x1874671477129088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830840.525341:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6776d00 x1874671477129088/t0(0) o38->lustre-MDT0001-mdc-ffff9a78a6512000@192.168.203.140@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830840.525348:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788849e800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830840.525351:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830840.525354:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6776d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477129088:192.168.203.140@tcp:38:kworker.0 00000100:00100000:0.0:1787830840.525360:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477129216:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830840.525396:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830840.525409:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830840/real 1787830840] req@ffff9a78a6774780 x1874671477129216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830840.525420:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a6774780 x1874671477129216/t0(0) o8->lustre-OST0000-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830840.525429:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830840.525433:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830840.525436:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a6774780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477129216:192.168.203.140@tcp:8:kworker.0 00000100:00100000:0.0:1787830840.525442:0:10933:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477129344:192.168.203.140@tcp:8:kworker.0 00000800:00000100:0.0:1787830840.525462:0:10933:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.140@tcp 00000100:00100000:0.0:1787830840.525484:0:10933:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830840/real 1787830840] req@ffff9a78a67761c0 x1874671477129344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787830840.525496:0:10933:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9a78a67761c0 x1874671477129344/t0(0) o8->lustre-OST0001-osc-ffff9a78a6512000@192.168.203.140@tcp:28/4 lens 520/544 e 0 to 1 dl 1787830865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787830840.525504:0:10933:0:(import.c:65:import_set_state_nolock()) ffff9a788923f000 lustre-OST0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787830840.525507:0:10933:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0001_UUID on 192.168.203.140@tcp failed (-110) 00000100:00100000:0.0:1787830840.525510:0:10933:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9a78a67761c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:97196825-2cb9-49ba-a47d-1455a997ce3a:10933:1874671477129344:192.168.203.140@tcp:8:kworker.0 00010000:00010000:1.0:1787830840.718897:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787830841.742405:0:48517:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830842.767124:0:48518:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830843.787381:0:48516:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 77452 lines, 77452 kept, 0 dropped, 0 bad.